On Delay Compensation & Recommendations for Routing

Ardour 6 Delay Compensation & Recommendations for Routing

Now that the dust has settled on Ardour-6.0 release, I’d like to point out some details regarding latency compensation.

General rules for routing

  • Prefer aux-sends over direct connections.
  • Avoid one-to-many direct connections (many-to-one is fine).

This article details on why and when following those rules is important.

Background

While developing full-graph latency compensation for Ardour 6, we faced one big issue: JACK. There were even times when we considered dropping JACK support.
One of JACK’s main features is that it facilitates anywhere-to-anywhere routing. Like an analog patch-bay one can use wires to connect everything without any constraints, even nonsensical connections.

“JACK provides mechanism, and does not enforce policy.”

However, unlike analog effects processors, some digital signal processors, do introduce a delay. Common examples are limiters, and pitch-shifters. Those effects need context, and hence buffer the audio internally which results in a delay. Still JACK allows one to arbitrarily connect those FX.

So let us have a look what happens when those effects are directly connected, using a latent effect as example:

Demonstration of effects of short delays (without latency compensation)

  1. Create an audio track. Its output is automatically connected to Ardour’s master-bus.
  2. Create an effect-bus. It is likewise auto-connected to Ardour’s master-bus.
  3. Add a latent effect to the Bus. This delays the signal passing through the bus.
  4. Connect the track’s output to the effect-bus input (in addition to the master-bus).

Figure 1 shows this as schematic. The direct signal from the track is summed with the delayed signal from the effect processor. Figure 5 below depicts a screenshot of an actual Ardour session with this setup.

Fig.1: block diagram
Fig.1: Example connection diagram showing ambiguous latency though different signal paths; red wire shows the signal path that has been delayed.

Sine wave phasing

When a sine-wave sample is loaded to the track and played while ramping up the delay of the effect-bus, the following can be observed:


Anim.1: Phasing of a 1kHz sine-wave, x-axis show time, y-axis signal amplitude, the delay time is in samples. green: source signal; red: delayed signal; blue: sum of the signals

As can be seen in this short video sequence, a 1kHz sine-wave when summed with a delayed version of itself cancels out after a delay of 24 samples (at 48kHz sample-rate).

Keep in mind that most latent effects have a fixed latency. The animation shows a variable delay for didactic purposes.

Comb filter effect

This can be generalized for all frequencies by looking at the spectrum of white noise. Different frequencies cancel out at different delays. This effect is called “comb-filter” after the resulting pattern in the spectrum:


Anim.2: Comb filter effect produced by summing white noise with delayed signal of the same noise

This demonstrates nicely why plugin delay compensation is important.

When using latent effects without delay compensation, some frequency bands in the resulting sound are notched out. If the delay is long enough, it may also lead to an audible delays or echo.

“Delay in the wire”

The above example shows a situation in which latency compensation is not possible.
This is because there there are two direct connections with different latency connected to a common destination.

The solution to this is to also delay the bypass accordingly:

Fig.2: Delay in the wire
Fig.2: “Delay in the wire” for latency compensation

In order to do this, a “delay in the wire” would be needed, however JACK does not offer that.

This is the reason why one should avoid one-to-many direct connections.

Ardour allows to seamlessly change back-ends. One can switch from JACK to Pulseaudio to ALSA (or ASIO, Coreaudio), to Dummy and back. So the constraint imposed by JACK’s design also extends to the connection logic of Ardour’s other back-ends. This is the reason why other Ardour backend likewise must not feature a delay in the wire (and why we’ve considered dropping JACK support).

Aux-Sends

This is where aux-sends come into play. Those offer routing that is internal to Ardour, and hence is not constrained by the backend.

As shown in Figure 3, aux-sends in Ardour include internal delay-lines, both for the send as well as the thru path. This allows to handle situation in which case a latent effect is either on the source-track, or the receiving bus.

Fig.3: Schema of internal delay-lines in a send
Fig.3: Schema of internal delay-lines in a send

As an added benefit aux-sends have a separate gain-stage and optionally an independent panner. This makes them generally more useful in most mixing situations.

Aux-sends can be added using the context-menu (right-click) in Ardour’s processor-box.

Fig.4: Adding an Aux-Send
Fig.4: Adding an Aux-Send

Ardour GUI Showcase


Fig.5: Screenshot of the Ardour Mixer, showing ambiguous latency due to connections (left) and delay-compensated routing using an aux-send (right)

The possibility to use both direct-connections and sends allows for nonsensical connections (this is also true for analog patch-bays), and situations that can lead to ambiguous latency. In Ardour a warning about this is displayed in the toolbar widget (Preferences > Appearance > Toolbar > Display Latency Compensation Info):

Fig.6: Ardour GUI Latency Indicator
Fig.6: Ardour GUI Latency Indicator

Conclusion

In summary this article explained in which case direct connections can lead to ambiguous latency, and how to avoid those by using internal aux-sends.

There are still many valid cases where direct explicit connections are preferable and valid.
In general this applies to all connections that have a single destination. e.g. Track to Master, or fanned-out multi-channel instruments.

This short article skips over major parts how external signals are aligned and read-ahead/write-behind techniques used to minimize overall delay.

Readers interested to dig deeper can find more information about this in the Thesis on Latency Compensation and Anywhere-to-Anywhere Signal Routing Systems.

19 Likes

So, just to be clear, this is only referring to instances when folks utilize JACK or are these recommendations whatever the backend being used?

The latter. It applies to all backends.

Since you can seamlessly switch between backends. All internal backends must have the same limitation as JACK (see also the “Delay in the wire” section of the article).

OK, thanks for the clarification and reminder to read the text very carefully :wink: It’s somewhat heavy reading pre-lunch if that’s any excuse!

So to take a specific example that I’ve used myself: Say I want parallel compression…I would normally add a compressor to a bus (maybe introducing delay due to using “lookahead” mode). In order for parallel compression to function as intended I would be routing the original track direct to master and also into the bus “with latent effect”. Correct me if I’m wrong but this is now resembling your Fig. 1. Are we saying that in order to follow the recommendations I still need to apply the same compressor plugin to the track with same lookahead but at 1:1 compression? I had assumed with Ardour 6 that this would be a thing of the past.

Hopefully I’m totally off base here and that my ramblings will maybe at least help clarify for others.

You have two options:

  1. Use a send to the bus with the compressor
  2. Use the effect inline in the mixer-strip via pin-connections (followed a summing plugin). Pin connections and bypasse are also latency compensated.

Which is why I wrote this blog post. There are cases where latency compensation is conceptually not possible, and JACK has always offered plenty of ways to shoot oneself in the foot.

Ideally we would not allow for any direct internal connections (and enforce using sends – like Mixbus does), or like other DAWs only expose physical I/O and hide all internal connections.

2 Likes

Thanks. This may explain my confusion given I’ve been doing most of my projects in Mixbus while I waited for Ardour 6.

1 Like

Thanks a lot, Robin! That is a very important post!

I was already wondering why I suddenly had “ambiguous latency” when trying out all kinds of stuff…

Can we then conclude that one is 100% save, as long as they don’t get this red “ambiguous” warning sign?:
ambiguous_latency_indicator

1 Like

Yes, but I’d say “should be” – give or take some unknown bugs :bug:

1 Like

Thank you very much Robin for this insightful post!

As I understand, if all the routing is internal, then the latency compensation should perfectly work, even if Jack is used as backend. So, what if there would be a setting in Ardour to make all the routing internal? People who want to use Jack with external connections could then just change this setting and would still be happy I guess. Did you consider something like this?

1 Like

This is similar to what Mixbus does. The master-bus has no input ports and can only be fed by internal sends.

We still want to allow direct outputs from tracks so you can use Ardour as tape-machine without master-bus. Also external sends in tracks need to remain a possibility.

So If all routing was done internally, there would still be need a way to allow users to add JACK ports on demand. Overall this would only complicates things, and still would not prevent a user from making nonsensical connections, either. In general we try to avoid nannying a user with “you must not do this”.

No, we have not considered a toggle-setting. If you could en/disable it in the middle of a existing session that would make things even more complex. Perhaps we could offer a one-way setting: only allow sends and when a user makes the first direct track -> bus connection, show a warning. That’s nannying, too, though.

1 Like

Thanks for clarifying. I didn’t understand the full complexity of the issues before. I understand and appreciate that Ardour avoids “nannying” users.

So, then the remaining “clean fix” for the situation that I see is to add that “delay in wire” feature to JACK. I am pretty sure you at least considered it and the decided that this won’t work for some reason. Could you elaborate on that? Or should I just read your thesis? :smile:

That would break JACK and existing JACK applications, as well be against jacks architectural design:

JACK provides mechanism” - data and latency information is provided, but jackd does not interfere.

<details>
There is a JACK API to set and query port latencies and inform applications about it. This works amazing well, for linear graphs. Also for direct one to one connections, JACK uses zero-copy shared memory. If there was a delayline in between that would no longer be possible. JACK won’t easily scale to thousands of ports anymore.

Anyway. I think the mistake was to use JACK for internal connections. All connections from Ardour to Ardour itself should not be exposed. – The problem with is that JACK started its life as engine for Ardour, and only later became a separate project.
</details>

I think for JACK this doesn’t need to be fixed, nor should it be. JACK is a patchbay.

JACK Applications should be viewed like blackbox monolithic devices (with lots of IO). You almost never connect those in ways that leads to ambiguous latency.

e.g. do not build a modular-synth with JACK, but you do use it to connect a modular synth JACK app to a JACK recorder app. – You connect MIDI input to Pure Data, Pd to Ardour, and Ardour to a ambisonics decoder… can be hundreds of channels.

1 Like

I am somewhat confused with this.
the pipe is internal but I can still see the connections with carla When working with jack

That’s exactly the (most common) source of the problem: in JACK, even internal Ardour->Ardour connections are visible and shown as normal virtual JACK wires between JACK ports, which means that the JACK routing and mixdown mechanism is used. Which implies that there cannot be latency “in the wire”.

<advanced>
Note that in fact you can still create ambiguous latencies with JACK, even if all Ardour->Ardour connections are hidden from it and have suitable delays “in the wire”: say you have two interfaces, one for the “control room” and one for the players to audition themselves. Say that these interfaces have a slightly different latency (even if world-clock synced, an usb interface and a pci one would not have the same latency). If you connect master to both of them at the same time, then JACK will advertise “ambiguous” latencies.

You could fix that by making ardour create a different output port for each “outgoing” connection, but that would probably be very confusing for the user (and that would prevent you to use any other « patchbay » jack connection gui, because they wouldn’t trigger the « spawn a new output» behavior of Ardour, and any connection made through them would be lost when the « dynamic » ports would be removed when Ardour deems they are no longer necessary).

</advanced>

1 Like

Only aux-sends are internal.

1 Like

Also note that JACK (non-internal) sends could also help get rid of the “ambiguous” latency, just because they sidestep the « one-to-many » problem since sends are a new port different from the track output.

Internal aux-sends are preferred though, since they avoid cluttering the JACK ports/connections. And external sends can (always ?) introduce a full period of extra latency (properly compensated for, since there is no ambiguity unless you connnect that send output to many destinations).

2 Likes

I thought that the only visible connections for the connector would only be external input and output buses
Sorry I have difficulties with the English language

There were even times when we considered dropping JACK support.

:frowning_face:

Please consider exposing fewer inputs and outputs and supporting fewer use cases before dropping JACK support altogether.

Readers interested to dig deeper can find more information about this in the Thesis on Latency Compensation and Anywhere-to-Anywhere Signal Routing Systems.

Thank you for publishing it. I was a bit sad at first, then I realized that only the first 50 pages are in French. :slightly_smiling_face:

That is unlikely to happen now. It was initially considered (put all options on the table), and then decided against. Some extra work was required to keep JACK, and make it work correctly when possible. IMHO it was worth it.

As @_FrnchFrgg elaborated, even if we hid all internal routing while keeping the current flexibility, possibility of direct outputs and arbitrary connections there would still be ways to create ambiguous situations.

With an analog patchbay in a studio one can likewise be creative with connecting gear, and in some cases you may not care about the artifacts, but even want them.

French Universities require that every thesis has at at least 50 pages in French, also those need to be first. It’d make a lot more sense if it was, say, 10% instead of fixed page count, but I am glad that it was possible to research and develop this at a University.

1 Like

Thanks for pointing out how to disable PDC. Adding and removing netjack connections would throw a gang of latency into the session.