CoreAudio / Axe Fx USB / Ardour = High Latency

I’m trying to record with my Axe Fx 3, USB connected on a Mac OSX.
It only support 48khz.

On the audio initial setup screen, it defaults to 128 samples (2.3ms latency).
But when recording, there’s a latency of… let say… 500 ms!

Ardour 5 has the ability to auto-detect latency, but it requires to physically connect output to input, which is not possible since this is virtual ins/outs. I could do it using JackCTL, but i think CoreAudio is the preferred way of getting it work on Mac OSX.

I also made a try syncing the AxeFx metronome with Ardour Tempo, which was very much more accurate than my guitar sound. This is weird…

Does anyone has any experience with a similar setup?