The "ShaleCrow" Project - progress and questions

Community Ideas for Ardour
#1

Hello all,

I’ve been working on some software for use with Mixbus and a multi-touch display. The working title is “ShaleCrow” (as in, “not a slate raven”).

I started by buying an Acer T272HUL 27-inch multi-touch screen; it’s HD 1440P (2560 * 1440 pixels) and was cheap ($250 used). It as an HDMI video input, and USB touch output. It can be used with the cross-platform UPDD device driver to send OSC messages to a program running on the computer, just like TouchOSC and similar programs on tablets.

Like TouchOSC, the ShaleCrow translator uses a “map” of the screen in which rectangular regions are defined by their location and size, and by the OSC messages that they send to Mixbus. ShaleCrow then listens to the messages coming in from the touchscreen, translates them using the map, and sends the corresponding OSC messages to the DAW. It’s fully multi-touch, meaning that you can drag several sliders at once, limited only by the number of fingers you have.

ShaleCrow also defines a control panel with buttons at the bottom of the screen; this includes transport controls, a jog wheel, and a host of other useful operations (see Figure 1 below).

There are options (controlled by the monitor screen in Figure 4) to display the region map overlaid on the Mixbus screen; examples of this are shown in Figures 2 and 3. There are mapping options for “large faders,” whereby the entire region at the bottom of the channel strip is used for the fader, and “details” mode, wherein all of the controls (pan, trim, mute/solo, compressor, etc.) are mapped.

Since many of the controls are too small to use with your fingers, I made large pop-up details screens for the strip controls (the top-most section of the strip), the EQ section, and the sends. Figure 5 below shows an example of the EQ pop-up.

I made a quick video demo of the working prototype for your comments; here it is:
https://vimeo.com/553581133

Notes

For the most part, ShaleCrow would also work with Ardour, though of course the screen layout is different; it does have a more complete OSC implementation than Mixbus.

The UPDD device driver supports many other touchscreen models, and supports multiple touchscreens on one computer, so it’d be possible to have both main DAW views responding to touch commands.

I’m looking for collaborators who can contribute design ideas and/or coding help to this project. I haven’t decided whether to make it open-source, closed-source freeware, or a cheap commercial product.

ShaleCrow is written in C++ and uses the JUCE and CSL frameworks, so it ought to work on Macs, PCs and Linux hosts.

Comments/questions are solicited…

Issues

As of now, the DAW screen layout is stored in an XML file that’s set up for each session (a terrible idea). It would be much better to either (a) “read” the screen (via computer vision software like OpenCV) to discover where the inputs and bus strips are, or (b) (DO THIS) augment Mixbus to respond to an OSC query with a list of geometry data, like the x-coordinate for the left edge of each channel, and the status of its strip (EQ and Sends expanded or collapsed). With MB32C, the channel widths and heights of the elements are fixed.

In theory (according to the Ardour manual) it’s possible to ask the DAW for the list of a channel’s plug-ins and their controls, and even for a list of the sends from a channel, but this appears not to be implemented in Mixbus. To make the pop-up editors work, I’d need to be able to get the send levels, EQ settings and other parameters from the DAW.

This isn’t rocket science, but I’m hoping for a partner in the project. I have the Ardour and Mixbus sources, and have most of the required libraries installed.

Figures

Figure 1. Mixbus DAW mixer screen with the ShaleCrow control panel visible at the bottom.
http://fastlabinc.com/screen-w-control-panel.jpg

Figure 2. Mixbus DAW mixer screen including the map overlay with rectangular regions mapped to OSC messages. This map uses large areas for the faders.
http://fastlabinc.com/screen-w-big-faders-map.jpg

Figure 3. Mixbus DAW mixer screen including the map overlay. This map uses shows the details for the controls for the channel and compressor.
http://fastlabinc.com/screen-w-detail-map.jpg

Figure 4. ShaleCrow monitor view showing its controls and log of OSC messages sent to the DAW (dragging 3 faders).
http://fastlabinc.com/monitor2.jpg

Figure 5. EQ detail panel.
http://fastlabinc.com/eq-details.jpg

Stephen Travis Pope
Santa Barbara, California, USA

See also https://forum.harrisonconsoles.com/thread-10174-post-56710.html

Mac library build: gtk+-2.24.23 fails?
#2

Wow that is quite a project!

A few years ago I tried something similar, directly built into Ardour using /dev/input/ for multi-touch on Linux, but gave up on it for various reasons.

I suppose it would be possible to implement a “get controllable at screen-pos (x, y)” method, that would go a long way.

–

… meanwhile looking at your control-panel screenshot. You may want to look into the icon tool in ardour’s source-tree. That can be used to generate scaled version of the buttons e.g:

553×553 40.7 KB

#3

Thanks for the suggestions, Robin!
Rather than the “get controllable at screen-pos (x, y)” solution, I’m thinking it would be easier on both ends to have the DAW simply dump a data structure that has the left edges of the channel strips along with a few details like the strip type (and for Mixbus, the status of the EQ and send panels). Other call-backs could retrieve the settings of the EQ, the sends and other details for the pop-up editors.

#4

My thinking behind wanting the overlay app to have a map of screen regions, rather than querying the DAW for the element at a given location, is that there are many screen elements that are too small to be accessed with the finger, so I thought of using pop-up detail views. Another option would be to have a “magnifying glass” effect when you press the screen and it’s not on a known large control (like a fader).
I’m getting all set up to be able to build Ardour/Mixbus, and will then have a closer look at these options.

#5

In the hope of getting some feedback, here are today’s default button panels, left and right halves, for suggestions.

Left

[Image: panel3L.jpg]

Right

[Image: panel3R.jpg]

#6

So I had started to look at buying this exact monitor to run the mixing board on and then just build the mount into the desk. What’s to keep you from undocking the external mixer, putting that on the external monitor, and then just using that to manipulate the parameters on the DAW directly without an OSC intermediary layer?

2 Likes
#7

The GUI toolkit that Ardour uses (GTK+ 2.x) cannot support multitouch.

#8

In the interest of long term design, I’d advise against the “jog” control. If nothing else, call it “shuttle”. At some point between now and 7.0, we’ll be doing a thorough vetting of “speed controls”.

#9

As Paul writes below, ShaleCrow doesn’t talk directly to the app other than through OSC, so the App doesn’t know about multitouch.

#10

Will do right away, Paul!

#11

Hello all,

After over 2 years away from the project, ShaleCrow is back on my front burner, and I have several ideas how to make it simpler.

I checked out Ardour 7 srcs onto a MacStudio running Ventura (13.5.1) and am re-implementing the changes I made before in the direction of getting a “get controllable at screen-pos (x, y)” method running.

Has anyone else building on a Mac run into a problem with the calls to new_fvec() e.g., in the VAMP plug-ins onset detector? This seems like old Intel code.

Any ideas?

Stephen, now in Ojai/CA/USA

#12

Are you still here?
Very interested.

#13

Very much still here and working on the multitouch DAW project.
The project is now called MixbusMultiTouchSupport (MMTS), and is very much under-way. I’ll post more screens and videos after Mixbus V10 is released in a couple of weeks.
stp

2 Likes