The longest delay is usually the initial time spent to scan sessions for recent list (if some of those session files are huge e.g. > 100MB).
Preferences > General > Engine > Try to auto-launch audio/midi engine
That will use the most recent settings when loading a session. However for new sessions Ardour will always ask what sample-rate to use. But just pressing < enter > here to accept the most recent setting is quick.
One way out is to use JACK. If jackd is running and you’ve used jack in the past, Ardour will use it unconditionally.
2 sec is realistic, but 5? for a new empty session? What CPU and disk does that system have? Perhaps Preferences > Plugins > VST > Scan for [new] VST Plugins on Application Start
is enabled? That may add a few seconds if you have many VSTs.