Audio in other software stutters if Ardour open

I’ve upgraded Ardour to 7.0 (and recently 7.1) and I’ve noticed that since then if I have Ardour open (nothing playing) the audio of any other software on my computer (firefox, vlc) stutters before it plays audio. This didn’t occur on Ardour 6.x

I’m running Ubuntu 22.04 with Pipewire but I did also notice this on 22.10 as well.

No idea where to begin diagnosing this!