Improving (MIDI) control surfaces using selectable modes

[matthijs1 (matthijs)]

I’m setting up Ardour for use with my X-touch compact control surface, and am running into some limitations that I would like to resolve. In this post, I’ll share some thoughts on this, and hope to get some feedback and maybe suggestions.

Background

I’m new to Ardour, just starting my first project, which is making a nice mix from a live recording of a choir/musical group performance, including both spoken scenes and songs. I have 21 channels, each individual singer/actor is separate, so I suspect my workflow might be a bit different than the typical usecase of a small band with diverse instruments, or producing electronic music. In particular, I suspect there will be a lot of repetition in my work, which makes it particularly useful to be able to use my X-touch as much as possible. However, since I’m not very familiar with Ardour, I might also be hung up on a particular work flow I think I want, even when that might not be the best approach in practice (so feel free to call me out on that). I’m an embedded software engineer by trade, so I have a tendency to add code to make things do what I want (I’ve already dug around the Ardour codebase a bit this week…).

Intended workflow

So, the workflow I imagine is similar to what the Mackie Control mode in Ardour offers:

1. Use banks to access all channels, with faders for the channel fader gain, buttons for solo/mute/select.
2. My X-Touch Compact (XT) has one encoder per channel, so I would like to change its function using other buttons (i.e. switch between channel trim, panning, FX send, etc.).
3. For controlling plugins, I imagine using the 8 extra encoders to control plugin parameters on the selected channel. Since there are more parameters than I have encoders, I would select the set of parameters to edit using one of the encoders or buttons (i.e. press one button to select EQ band 1, another to select EQ band 2, one more for the compressor), etc.

AFAICS I cannot achieve this with the MC mode, because the MC mode does not offer editing plugin paramters, and also the MC mode on my XT does not send the right MC commands for this anyway (which could be remapped on the XT side, but still prevents editing plugin params AFAICS).

Using the Generic MIDI control surface in Ardour, I think I can express every control (provided I use a consistent plugin ordering, since identifiying plugins by name is not possible), except that I cannot seem to use the same encoder for different purposes. The Sn/Bn strip numbering does allow for using a single encoder to control different tracks, but I cannot see a builtin way to switch one encoder between e.g. EQ band 1 freq and EQ band 2 freq.

Implementation

So, I’m considering some approaches to make this possible:

1. Modify Ardour Generic MIDI code to support “mode variables”. I can imagine entries in a MIDI map could either set a specific mode variable (possibly in addition to, or instead of a normal action/uri/function), other entries could then be tagged to only be active in a specific mode (plus some code to handle feedback when switching modes). It’s probably quite a bit of work to get this working quickly and not as flexible as some of the other options below (especially relevant while I’m still exploring what I need exactly).
2. Adding a LUA session script that essentially implements a control surface driver by reading MIDI and applying the changes directly. Upside of LUA is that you can potentially control more than what is exposed through OSC/Generic MIDI, but this does mean reimplementing a lot of things (feedback, banking, etc.). I’m also worried a bit about the development experience - reloading and debugging sessions scripts did not look very convenient at first glance.
3. Running an external service (probably a python script) that reads MIDI and controls Ardour through OSC. This is probably a lot more convenient to develop, and reuses stuff (feedback, banking) from the OSC implementation in Ardour. Specifying the mapping, including mode switches, can probably be generalized with a bit of YAML or JSON.
4. The same, but using Open Stage Control with a custom module to translate MIDI to OSC and back. This might be even easier, since Open Stage Control already handles MIDI and OSC connections, and I can just implement the right filters. As an added advantage, it is probably easy to add some additional UI (i.e. show the current bank and mode).
5. Using an external MIDI translator, such as mididings. This would map all possible controls in the Ardour MIDI map to different MIDI messages, and then use mididings scenes for modes, mapping the same encoder to different midi messages based on the current mode/scene. Downside of this approach is that it splits the mapping configuration between Ardour and mididings, which is not so nice.
6. The same, but using x42 MIDI filtering plugins inside Ardour to do the mapping. This keeps everything inside Ardour, but probably results in a complex stack of plugins that control themselves, so probably more complex than splitting the config between Ardour and mididings (if it is at all possible, I’m not sure).

Right now, I’m inclined to try option 4, using Open Stage Control to map my XT to Ardour’s OSC messages. Given the mapping will be done by a bit of my own custom code, this also gives extra flexibility to implement extra things (possibly hardcoded) I might figure out I need later on. Does this make sense? Or are there other ways that I might have missed?

Unifying control surface implementations?

Looking a bit around the control surface implementations, I did also wonder if it would not make sense to unify them more. E.g. MC surfaces are (or at least can be, I think) essentially just sending MIDI, so if the MIDI mapping was made sufficiently expressive, a separate MC implementation would not be needed at all. Also, the Generic MIDI uri/action/function specifiers are largely the same (i.e. they can express the same things) as OSC addresses, so it would make sense (to my outsider newbie mind) to just map MIDI messages into OSC messages, making things more uniform and potentially easier to maintain (and exposing new things benefits more surfaces then instead of having to duplicate work). I can imagine this is not currently the case because of how things evolved and the effort of making the conversion now, or maybe there are technical reasons to not allow this.

[paul (Paul Davis)]

I can’t comment on this as much as I would like, mostly because I’m currently travelling. However, regarding the last section, I will note that one the main problems with control surface support is maintaining the state mapping between the surface (e.g. lit buttons, colors) and ardour. It is quite hard to generalize this. Even different Mackie Control Protocol devices do not work in precisely the same way when it comes to this stuff.

The Generic MIDI support essentially lacks any support for “the surface has state”, unless the surface meets a very stringent requirement (that the state is set by the same messages that affect ardour state).

[matthijs1 (matthijs)]

@paul

I will note that one the main problems with control surface support is maintaining the state mapping between the surface (e.g. lit buttons, colors) and ardour. It is quite hard to generalize this. Even different Mackie Control Protocol devices do not work in precisely the same way when it comes to this stuff.

Right, that’s a good point. In theory, this would be solvable with a sufficiently expressive mapping description format, but I can imagine that you then either end up with either an impossibly complex format, or something that’s essentially a programming language.

It might be interesting to explore such a more expressive mapping description, which is what I’m actually currently doing (see below). I will probably not be able to see that through for all the devices that Ardour currently supports, but I’ll see how far I come for my control devices (X-touch compact and I-controls pro), and maybe the result can be a starting point for integrating something like it in Ardour.

@matthijs1

Right now, I’m inclined to try option 4, using Open Stage Control to map my XT to Ardour’s OSC messages.

I’ve started along this route, and as expected, it was indeed quite easy to get something basic going. In an hour and two dozen lines of javascript code I could control the master fader from my x-touch and have feedback. Also, open-stage-control turned out to support auto-reload of the javascript code, which is really convenient during development.

I’ve pushed out my code to github for anyone that is interested:

GitHub - matthijskooijman/MIDI-OSC-Mapper: Module to map MIDI messages from a...

By now, I’ve expanded the code to support more flexible mappings. In particular:

• I’ve used the nerdamer math javascript library to flexibly map any arguments in a MIDI or OSC message into variables, possibly modifying them along the way. For example, the first channel rotary sends a midi message with channel=10 (the first 9 channels are the faders), which in the mapping is matched against c+9, which uses equation solving (c+9=10) to derive c=1, which is then used as the channel number in the OSC message to ardour. For feedback messages, the reverse happens (Ardour sends 1 as the channel number, which is mapped against the c variable, and then for the midi message c+9 is evaluated to get midi channel 10. The same generic approach is currently used to handle scaling of values (e.g. 0-127 → 0-1). The nice thing of this approach is that you only need to specify the conversion one time, not both ways separately. Also, it allows a single mapping to be used for all channel faders or rotaries, no need to specify each separately.
• In addition to mapping to OSC messages, controls can also set global mode variables. Each mapping then has an if property that can be used to activate the mapping only in a certain mode. This is still quite rough - I’m not quite sure how I want these modes to be mapped on my surfaces in practice. Also, feedback does not work properly yet for these conditional mappings (I would need to remember previous feedback and reapply it after switching modes, or trigger ardour into re-sending feedback after every mode change).
• Because this is implemented generically, you could also use global variables for other things, like banking (if you do not want to use Ardour’s banking code) or storing plugin parameter offsets, etc… Additionally, the if clauses are now also used to pose limits on derived variables (e.g. to have the channel fader mapping only apply for channel 1-8 and not also on channel 9, which is the master fader, allowing the master fader to be separately mapped).

As an example of the above, here’s an excerpt from the mapping definition in my script:

        {
            // Master fader
            'from': ['/control', 1, 9, 'v'],
            'to': ['/master/fader', 'v / 127'],
        },
        {
            // Channel faders
            'from': ['/control', 1, 'c', 'v'],
            'to': ['/strip/fader', 'c', 'v / 127'],
            'if': ['c => 1', 'c <= 8'],
        },
        {
            // TODO: Feedback for coniditional mappings
            // Channel rotaries
            'from': ['/control', 1, 'c+9', 'v * 127'], 
            'to': ['/strip/trimdB', 'c', 'v * 40 - 20'],
            'if': ['c => 1', 'c <= 8', 'channel_mode == TRIM'],
        },
        {
            // Channel rotaries
            'from': ['/control', 1, 'c+9', 'v * 127'],
            // TODO: This seems to invert (0=R, 1=L)?
            'to': ['/strip/pan_stereo_position', 'c', 'v'],
            'if': ['c => 1', 'c <= 8', 'channel_mode == PAN'],
        },
        {
            // Extra rotary 9 button
            'from': ['/note', 1, 8, 127], // 127 = press, 0 = release
            'set': {'channel_mode': 'TRIM'},
        },
        {
            // Extra rotary 11 button
            'from': ['/note', 1, 10, 127], // 127 = press, 0 = release
            'set': {'channel_mode': 'PAN'},
        },

As an idea for further improvement, I’m considering splitting the mapping into two (or maybe three) parts: One mapping that maps the MIDI messages into more semantic names (e.g. /fader 1, or /rotary 3), with values normalized to the 0-1 range, possibly also explicitly association touch messages with the associated fader, or links pressing and rotating the same encoder), and then a second mapping that mapps the semantic control names into Ardour functions/OSC messages. Maybe the second one could also be split by introduced slightly more semantic names, in a third mapping step, so the middle mapping step is easier to define/read. However, Ardour OSC names are already pretty readable, so I suspect this would be mainly useful for plugin parameters (which currently are quite cryptic).

In addition, I currently have values mapped in a generic way (the script does not know which are control values and which are “metadata” like channel numbers), which is elegant, but also prevents remembering the last value of feedback received from Ardour (needed when switching modes), since we need to remember the last value of a given property, not just the last set of values for a given address. So splitting into multiple steps, normalizing the value into 0-1 range probably also provides some opportunity for splitting of the value (it probably always is the last value in the OSC message, but I’m not sure that’s safe to assume. Having an intermediate semantic format that is defined by ourselves might make this safe to assume - even if the MIDI incoming messages or OSC outgoing messages are different, the semantic messages could be made like that).