Polishing “Audio/MIDI Setup” window and latency calibration best practices

[agus.terol]

Hello, I think this is my first message in this forum, despite I dayly follow it and Ardour development. So I will introduce myself: I am Agus Terol from Alicante, Spain, and I have been using Ardour (as amateur/hobbyist musician) for several years now. I would say that from the 2.x series times.

First of all, I would like to thank all the developers their work and effort along these years, and congratulate them for all the success and achieved goals.

Recently, I have been focused on testing Ardour 6’s “Audio/MIDI Setup” window in order to find bugs and provide some feedback about its workflow. I am very excited about the complete latency compensation feature that Ardour 6 brings, and I know it has taken a long time to implement it. Also, in my opinion, if users do not proper calibrate the audio and MIDI devices, this feature could not work properly. That is the reason I am commenting here the need of polish that window, so that the users can receive better feedback about what Ardour is doing, and have a better experience, since this window appears after launching Ardour and opening/creating a session.

At the moment I have reported 6 issues (bugs and features/improvements). Please, ignore #8052 because I thought it was not created after being blocked for a while by the bug tracker.

• #8048 After calibrating MIDI, audio calibration might be required again
• #8049 “Audio/MIDI Setup” window does not wait until the user really wants to let Ardour load the current session after calibrating audio
• #8050 Add a “Cancel” and/or “Back” button in the “Audio/MIDI Setup” window
• #8051 Add a “Refresh devices” button in “Audio/MIDI Setup” window
• #8053 A MIDI device connected after opening “Audio/MIDI Setup” window appears when calibrating another MIDI device
• #8054 MIDI calibration does not produce expected results like Audio calibration does

My main goal is to have a MIDI track in Ardour that is sent to two external keyboards. Then, the audio signal of those keyboards is captured in two audio tracks. As far as I know, if MIDI and audio is correctly calibrated, the recorded audio signals will have a wave synchronized with the MIDI events. That’s partially true, because assuming that the MIDI message is sent a bit earlier by Ardour (due to the latency values got with the MIDI calibration), every device will take a time to process it and render the sound to its audio outputs. So, I assume it should be a delay there. Then, when the audio signals are captured by Ardour, they would be late, but the audio calibration values would make Ardour to move them forward in time.

For these tests, I left audio and MIDI cables used for calibration connected. So, I can record them on respective audio and midi tracks. In the case of the audio signal, I take as reference the metronome click signal, which is routed to an audio output port which is connected with a cable to the audio input port used in the recorded audio. That way, I can compare the delay of the other recorded audio signals. The metronome click signal is captured and perfectly synchronized (except under some circumstances, see bug #8048). The same technique is used with MIDI tracks. Below images illustrates this test, ran with Ardour 6 RC1-82:

As you can see in the image, I cannot achieve the desired synchronization. I am using a big buffer size of 1024 samples (and 3 periods for samples and 48 kHz) for making these errors bigger. Ardour reports 21.3 ms of latency. The audio signals are both delayed 9 ms. That value seems to me a bit high, because when playing the keyboards with headphones, I think I would appreciate that delay, but I am not sure.

However, I would have expected the MIDI signals to be not so much delayed (above 21 ms and 22 ms, respectively). Maybe there are bugs, or maybe I am wrongly understanding some concepts here and/or doing incorrect assumptions or performing wrong measurements. So, that is why I also wanted to discuss about best practices for audio/MIDI calibration. What can be achieve? What cannot be achieved?

I am sorry if this post is being very long, but I had several doubts and I did not know how to expose them atomically, because they are more or less related.

Best regards, and thank you!

[agus.terol]

PS: I think I was not clear enough. Since I am using audio and MIDI tracks, my post could also apply to Ardour 5.12 (because it has latency compensation for tracks), and Ardour 6 has gone beyond and has applied it to buses, sends, etc. However, I have read all the development updates, and I know that deep changes have also been done in the core of Ardour, and some MIDI bugs could have been fixed. I do not know if they could help in an scenario like this. But, anyway, we are testing Ardour 6 now!

[paul (Paul Davis)]

First simple question: did you calibrate systemic latency from the audio/MIDI setup dialog? (It requires a loopback cable to plug the output of your audio interface into its input)

[agus.terol]

Yes, I did, and because of #8048 I had to calibrate audio again after MIDI calibration.

[x42 (Robin Gareus)]

With some USB devices on Linux/ALSA the systemic audio latency changes (kernel side buffer) with every x-run. Could this explain the issue?

Here it’s very reliable, calibrating MIDI (ALSA Sequencer) has no effect on Audio at all.

[agus.terol]

With some USB devices on Linux/ALSA the systemic audio latency changes (kernel side buffer) with every x-run. Could this explain the issue?

It could, it is a USB device: Focusrite Scarlett 18i20 (1st gen). I am sure that it changes, because after stopping the audio engine, and starting it again, calibration is needed again, and it arises different values.

However, how can I know when an x-run has occurred if I am not recording (i.e. when calibrating MIDI)?

Here it’s very reliable, calibrating MIDI (ALSA Sequencer) has no effect on Audio at all.

I was calibrating MIDI using “ALSA Raw”. I will do the tests tomorrow using “ALSA Sequencer”. Anyway, I was also calibrating the MIDI USB from the same device than the audio (Scarlett 18i20), could then interfere in any way if it is the same USB device?

[agus.terol]

Hi, I have ran the same test with ALSA Sequencer but I have obtaining the same results.

However, I have discovered something interesting. When I have launched Ardour, and “Audio/MIDI Setup” window has appeared (with the red “Stopped” label referring that the audio engine is not turned on), I directly went to calibrate MIDI. After MIDI calibration, when clicking “Back to settings”, Ardour continued opening the session by previously starting the audio engine. So, I am not sure about the sentence “calibrating MIDI (ALSA Sequencer) has no effect on Audio at all”, because in some way is interfering.

I have added a note in #8049 because I think it is related, at least from the same point of the user experience.

[agus.terol]

Hi, I have done the same tests in another computer, and I get the same results.

However, I have been investigating and I have observed some interesting behavior when changing the buffer size, that I didn’t realize the first time I did these tests. The recorded notes from the armed MIDI tracks (see above image) have a delay that is proportional to the audio size buffer. For instance, with a buffer of 256 samples, I got a delay of 234 and 261 samples, and with a buffer of 1024 samples, I got 998 and 1098.

So, in some way MIDI is tied to audio, no? Also, switching the periods from 3 to 2 arose these results:

• 1024|3: detected roundtrip latency 3141 samples (65,4 ms); systemic latency 1093 (22,8 ms).
• 1024|2: detected roundtrip latency 2119 samples (44,1 ms); systemic latency 71 (4,5 ms).

However, the recorded notes are delayed by the same amount in both cases, no matter the number of periods. Do you guys (@x42 and @paul) think this could ring some bell? Like a off-by-one bug when Ardour is compensating MIDI according to the periods. Or this is totally unrelated?

[agus.terol]

Hi, recently I’ve done the same tests I mentioned at the beginning of this thread in another computer. This time I have used a PCIe RME RayDat, and let the Focusrite Scarlett 18i20 connected via ADAT in the role of a standalone A/D and D/A converter. The audio tracks show a better result now; they are quite near to the calibrated wave of the metronome click. I guess this can just be explained by the time the keyboards take to process the MIDI message and emit the sound (less than 2 ms), can it? Well, the Clavia is not calibrated (because I cannot do that), but I can do that with the Roland, since it has the option to enable “USB MIDI Thru Sw” (according its options menu), which allow to map the 5 pins MIDI connectors to the USB ports.

However, the MIDI tracks are still showing the same result. This time I am not using Scarlett’s MIDI ports; it is disconnected from the computer. But I have added to the tests the two RayDat’s MIDI port, which are connected with two cables (the same I used at calibration time).

If they keyboard’s sound is sync’d, that means that Ardour emits the MIDI message a bit earlier, the keyboard receives it and processes in time, and then the captured sound is moved forward by Ardour. So, why is this not working similar for only MIDI tracks? I still see something suspicious with the MIDI delay being very close to the buffer size (1024).

Recently I remembered the kernel’s 1000 Hz config, and I have discovered I have it set to 250 Hz. Do you think this can be the cause? I think I heard some time ago that this option is not required.

$ grep "CONFIG_HZ" /boot/config-5.6.0-1-rt-amd64
# CONFIG_HZ_PERIODIC is not set
# CONFIG_HZ_100 is not set
CONFIG_HZ_250=y
# CONFIG_HZ_300 is not set
# CONFIG_HZ_1000 is not set
CONFIG_HZ=250

[x42 (Robin Gareus)]

@agus.terol

Recently I remembered the kernel’s 1000 Hz config, and I have discovered I have it set to 250 Hz. Do you think this can be the cause?

No. Those timers are not used by Ardour nor MIDI I/O. Some ancient legacy apps made use of those timers (if and only if they had permissions to do that).

You should be better off with a tickless (NO_HZ) kernel. for kernel timestamps (alsa-seq), snd-timer uses high-resolution (hrtimers) if needed these days.

In any case those settings would only affect jitter, not latency.

[agus.terol]

Curiously, lsmod doesn’t show “snd-hrtimer” loaded. Anyway, if I modprobe it, it doesn’t change anything. But, do you think it would be good to add it to /etc/modules ?