Stupid User Tricks - AutoMixing multiple projects with a shell script

system · April 22, 2006, 1:25am

I have an oddball little thing I’m doing with ardour that may or may not be an idea anyone else is interested in, so I’ll share it.

But I’m also having a small problem with it, and am looking for a hand figuring it out. This could go under either “how do I” or here, I suppose… I’ll just post it here.

First, the scenario in which it would be useful… mine. Or maybe any other instance where one has a large number of ardour projects that consist of live recordings (multiple tracks) of the same sources.

For example, I started recording band rehearsals and writing/jam session. Each “take” is a separate named ardour project, but they all are identical sources. The same instruments, the same mics, the same levels on all of the amps/mixers/etc.

At the end of a session I would do a rough mix (setting basic levels for the tracks, etc), exporting to wav/mp3, then distribute to the other participants before heading home.

I found myself making the same kinds of changes for each file, which was sort of time consuming. I started thinking there had to be a way to automatically make the same changes to a bunch of ardour files at once.

(Perhaps making those changes to my template beforehand would get me the same end-result… but I still thought “mass identical changes” might be nice. Namely the ability to apply mass changes AFTER recording, which would be outside the scope of templates. I could also maybe record everything in a single long session… But thought it might be time consuming to break it back into pieces. Tips along those lines are welcome.)

Anyway, here’s what I’m doing… After testing last night there is a bit of a hicccup, but it is mostly working:

Using a shell script, some variable assignment (to handle track names and changes to make) and a bunch of string replacements (with sed), I’m making changes directly to the projectname.ardour file.

The first thing the script does is look for a configuration file. Which is nothing but another bash script that assigns a bunch of variables. First it looks in the current working directory, then in the home directory. If found in neither, it goes with some defaults.

Then it takes the name of the file to be processed (passed via command line of course) and goes to work making a bunch of string replacements based on the settings made in the config file. Pretty simple, really.

Here are some small snippets from it… Enough to get the basic idea:


#  Track Names:
track_00=master
track_01=d_kick_01
track_02=d_snare_01
#  Mute Setting:
mute_00=no
mute_01=no
mute_02=no
#  Invert Phase Setting:
phase_00=no
phase_01=yes
phase_02=no
#  Gain Changes:
gain_00=1.000000000000
gain_01=1.410375475883
gain_02=0.489784628153


sed -e "s/\(name=.$track_00.*\)gain.*iolimits/\1gain=\"$gain_00\" iolimits/" $filename |
sed -e "s/\(name=.$track_01.*\)gain.*iolimits/\1gain=\"$gain_01\" iolimits/" |
sed -e "s/\(name=.$track_02.*\)gain.*iolimits/\1gain=\"$gain_02\" iolimits/" |


#  Doing a match over 3 lines is a pain, and it was easier to
#  Combine two of the lines first, then break them back up again when done.
#  So that's what I did.
sed "N;s/\(Route.active.*\)\n\(.*MIDI.*\)/\1XXX\2/g;P;D;" |
sed "N;s/\(Route.active.*\)muted=.*soloed\(.*\)phase-invert=.*mute\(.*\n.*IO.name=.$track_00\)/\1muted=\"$mute_00\" soloed\2phase-invert=\"$phase_00\" mute\3/g;P;D;" |
sed "N;s/\(Route.active.*\)muted=.*soloed\(.*\)phase-invert=.*mute\(.*\n.*IO.name=.$track_01\)/\1muted=\"$mute_01\" soloed\2phase-invert=\"$phase_01\" mute\3/g;P;D;" |
sed "N;s/\(Route.active.*\)muted=.*soloed\(.*\)phase-invert=.*mute\(.*\n.*IO.name=.$track_02\)/\1muted=\"$mute_02\" soloed\2phase-invert=\"$phase_02\" mute\3/g;P;D;" |
sed "N;s/\(Route.active.*\)XXX\(.*MIDI.*\)/\1\n\2/g;P;D;" > $filename.automix

If anyone actually wants a copy of this thing, I’m happy to post it. The guts that make it actually work are right there, though.

FWIW, I’m sure an actual programmer (which I am not) could do a much better/cleaner job of it with better design (even I know it’s pretty ugly/hackish), or even by using proper XML processing tools instead of bash and sed. But hey… It’s a start and it is mostly working.

That said, here is the problem I’m having:

For some reason, ardour doesn’t seem to like opening a file that doesn’t end in the name “.ardour”. Even just copying/renaming a known good file to anything else seems to make it barf. (It says it can understand it, and won’t open it.)

It doesn’t like it if .ardour appears more than once, either. (“project1.ardour.new.ardour”) Oddly enough, if I have it just overwrite the original file (“project.ardour”), it works beautifully. It does exactly what I expect it to do. Mutes the track I want muted for all but one or two of the projects, flips the phase on the kickdrum track, sets the levels to match what I want. (I got the numbers by reading the file from a project with good basic levels.)

Another thing it doesn’t like… I have another small script whose sole purpose is to call this one against a whole batch of files. In my filesystem, all projects recorded on a given day are in unique directory, named by the date. My goal is to drop into this particular directory (which may or may not contain a unique config file to be used), and… well, just read it:


for file in */*.ardour
do
ardour_automix.sh $file
done
echo " "
echo "Press ENTER to continue. "
read continue

It works, but… ardour won’t read the resulting file. It’s weird… for the same file I can run the main script and it works fine. But call it with this little snippet, and it can’t read the file it makes. I haven’t done a diff on a copy of each yet (ran out of time) but… it seems silly that there would be a difference.

Am I missing something totally obvious? Is there some magic involved with ardour reading an XML file that makes it not like copy of a working file that just doesn’t match the usual name it uses? Is there some magic in an XML file to make some inexplicable difference between the result of running my main script manually (which works) and calling it with another one (which doesn’t)?
Any help with those last annoying bits is much appreciated!

Now, my ultimate goal will have to wait until ardour to gains some “offline” scripting ability. Namely, to export an entire project to wav (the main L/R channels, same as one can do in the GUI) via the command line.

That would be the slickest thing ever. If I could do that… imagine what could happen with a single script! The entire nights work could be rough-mixed, exported to wav, burned to CD, and encoded into mp3… all while I go have a beer. That would ROCK!!!

sampo · April 22, 2006, 12:12pm

It would be easier (imho), simpler and more correct to do this with an XML parser. Run the parser through the .ardour file, apply the configured changes and output to another XML file.

It would be a lot easier to analyze the issue with your script if we could take a look at an .ardour file produced by the script.

system · April 24, 2006, 5:55pm

You’re absolutely correct, I’m sure. (I think I even mentioned that.)

Before I started I spent a couple of minutes looking for a quick online tutorial/example or anything else to start down that route (XML processing) with. With no immediate luck, I quickly defaulted to something I knew how to do already. (And had bits of working scripts I could start with… just had to figure out the regexp was all.) Wanted to get the itch scratched quick. So much for that.

I had forgotten to bring home any examples with me; having wasted time/been distracted by an unrelated mistake. (The space and machine are kindof out in the boonies, lacking a net connection, and I have timited time before and after sessions to muck with the machine. Kindof why I’m trying to automate some of it. I intend to get an external HD soon, so I can bring it all home to my own workstation.)

I’ll be back out there this evening to try it again with a minor tweak, and to grab a bunch of files worth looking at and sharing. (I’ll post some unless I find a silly/obvious explanation that got overlooked in my hurry.)

In the meantime I think I’ll look further into learning proper parsing/manipulation of XML files. Do you have any reccommendations for tools and/or information resources to learn it from?

Thanks!

taybin · April 24, 2006, 7:13pm

Python and its minidom xml library make XML parsing pretty easy. It might help to already know python though.

system · May 3, 2006, 10:26pm

Thanks… I think that’s a route I’m going to go long term. Don’t know Python, so will have to start with that first.

But in the meantime, all I had to do to fix the script was to send all the changes to a new file first ($filename.automix as written above), then copy over the old one with it.

I had been directly overwriting the original filename at the end of all of those pipes. Worked fine one at a time, but en masse I guess it ended up chasing, catching, and eating its own tail. Something like that.

As of now it doesn’t do a few things I want yet (panning changes), but as it is it is saving me a bunch of time at the end of the evening.

It saves a copy of the original file, so you can backtrack if it goofs anything up. So it hasn’t caused me any trouble.

Anyway, thanks.

Here’s the whole thing (even uglier without empty lines/spacers, but the code tag here seems to automatically end whenever it encounters them. So I squashed it.)


#!/bin/bash
#************************************************#
#               ardour_automix.sh                #
#                                                #
# Changes gain levels in an Ardour project file  #
# for tracks and levels defined in a config file #
#************************************************#
# A filename should have been passed on the command-line.
# If not, complain about it.
if [ -z "$1" ]
then
  echo " "
  echo "Um... you forgot to specify a filename.  Sorry."
  echo " "
  echo "Press ENTER to continue. "
  read continue
else
  # If a filename was given, we can continue
  # First, look for a config file.
  # It is nothing but a list of variables containing tracknames and gain levels
  # to be used.  Check the "defaults" below to see what it should look like.
  # You can use more than one.
  # It'll look for one in the current directory first.
  # If none are found, it'll check the user's home folder.
  # If none are found in either location, it uses some defaults.
  configfile=ardour_automix.conf          # Name of the config file
  echo " "
  echo "Checking for config file:"
  echo "  Checking current directory..."
  if [ -e "$configfile" ]                 # Check for config file in pwd and use it if found
  then
    echo "Found it."
    . $configfile
  elif [ -e ~/$configfile ]               # Otherwise check in home folder and use it if found
  then
    echo "  Checking home directory..."
    echo "Found it."
    . ~/$configfile
  else                                    # If no config file is found anywhere, use some defaults
    echo "  Checking home directory..."
    echo "Not found.  Using defaults instead."
    #  These are default tracknames and gain levels to use if no config file is found.
    #  Plug in your track names and/or adjust them as neccessary.
    #  Open an ardour file with decent level settings in a text editor, find the
    #  relevant lines, and copy the gain settings here.
    #  Config files should look exactly like this... simple variable assignment.
    #  Remember to start any config file with a sha-bang, or those variables
    #  won't hang around long enough to be used.  Duh.
    #  Track Names:
    track_01=d_kick_01
    track_02=d_snare_01
    track_03=d_overhd_L
    track_04=d_overhd_R
    track_05=guit_han_01
    track_06=guit_mike_01
    track_07=bass_01
    track_08=voc_mike_01
    #  Mute Setting:
    mute_01=no
    mute_02=no
    mute_03=no
    mute_04=no
    mute_05=no
    mute_06=no
    mute_07=no
    mute_08=no
    #  Invert Phase Setting:
    phase_01=yes
    phase_02=no
    phase_03=no
    phase_04=no
    phase_05=no
    phase_06=no
    phase_07=no
    phase_08=no
    #  Gain Changes:
    gain_01=1.410375475883
    gain_02=0.489784628153
    gain_03=0.343691915274
    gain_04=0.343691915274
    gain_05=0.700249791145
    gain_06=0.700249791145
    gain_07=1.598044633865
    gain_08=0.641410887241
  fi
  #  OK, by now we should have the file to be worked on, and all the variables we need to do it.
  #  Grab the filename to be edited:
  filename=$1
  #  Now search for lines by track name... and change the gain.
  #  Make sure there is a line for each track, and that the variable names match.
  #  This bit is probably a candidate for cleanup.  Maybe looping through all of the variable
  #  names or something like that.  I'm not gonna bother, myself.  :)
  sed -e "s/\(name=.$track_01.*\)gain.*iolimits/\1gain=\"$gain_01\" iolimits/" $filename |
  sed -e "s/\(name=.$track_02.*\)gain.*iolimits/\1gain=\"$gain_02\" iolimits/" |
  sed -e "s/\(name=.$track_03.*\)gain.*iolimits/\1gain=\"$gain_03\" iolimits/" |
  sed -e "s/\(name=.$track_04.*\)gain.*iolimits/\1gain=\"$gain_04\" iolimits/" |
  sed -e "s/\(name=.$track_05.*\)gain.*iolimits/\1gain=\"$gain_05\" iolimits/" |
  sed -e "s/\(name=.$track_06.*\)gain.*iolimits/\1gain=\"$gain_06\" iolimits/" |
  sed -e "s/\(name=.$track_07.*\)gain.*iolimits/\1gain=\"$gain_07\" iolimits/" |
  sed -e "s/\(name=.$track_08.*\)gain.*iolimits/\1gain=\"$gain_08\" iolimits/" |
  # Let's flip the phase and mute settings on any tracks that need it.
  sed "N;s/\(Route.active.*\)\n\(.*MIDI.*\)/\1XXX\2/g;P;D;" |
  sed "N;s/\(Route.active.*\)muted=.*soloed\(.*\)phase-invert=.*mute\(.*\n.*IO.name=.$track_01\)/\1muted=\"$mute_01\" soloed\2phase-invert=\"$phase_01\" mute\3/g;P;D;" |
  sed "N;s/\(Route.active.*\)muted=.*soloed\(.*\)phase-invert=.*mute\(.*\n.*IO.name=.$track_02\)/\1muted=\"$mute_02\" soloed\2phase-invert=\"$phase_02\" mute\3/g;P;D;" |
  sed "N;s/\(Route.active.*\)muted=.*soloed\(.*\)phase-invert=.*mute\(.*\n.*IO.name=.$track_03\)/\1muted=\"$mute_03\" soloed\2phase-invert=\"$phase_03\" mute\3/g;P;D;" |
  sed "N;s/\(Route.active.*\)muted=.*soloed\(.*\)phase-invert=.*mute\(.*\n.*IO.name=.$track_04\)/\1muted=\"$mute_04\" soloed\2phase-invert=\"$phase_04\" mute\3/g;P;D;" |
  sed "N;s/\(Route.active.*\)muted=.*soloed\(.*\)phase-invert=.*mute\(.*\n.*IO.name=.$track_05\)/\1muted=\"$mute_05\" soloed\2phase-invert=\"$phase_05\" mute\3/g;P;D;" |
  sed "N;s/\(Route.active.*\)muted=.*soloed\(.*\)phase-invert=.*mute\(.*\n.*IO.name=.$track_06\)/\1muted=\"$mute_06\" soloed\2phase-invert=\"$phase_06\" mute\3/g;P;D;" |
  sed "N;s/\(Route.active.*\)muted=.*soloed\(.*\)phase-invert=.*mute\(.*\n.*IO.name=.$track_07\)/\1muted=\"$mute_07\" soloed\2phase-invert=\"$phase_07\" mute\3/g;P;D;" |
  sed "N;s/\(Route.active.*\)muted=.*soloed\(.*\)phase-invert=.*mute\(.*\n.*IO.name=.$track_08\)/\1muted=\"$mute_08\" soloed\2phase-invert=\"$phase_08\" mute\3/g;P;D;" |
  sed "N;s/\(Route.active.*\)XXX\(.*MIDI.*\)/\1\n\2/g;P;D;" > $filename.automix
  mv $filename $filename.orig
  mv $filename.automix $filename
  #  It's done.  So tell me about it.
  echo "Finished file -----------------------------> $1"
fi

And this calls it for a whole batch. (Needs to be from the parent directory containing the ardour project directories you want it to operate on.)


#!/bin/bash
#************************************************#
#             ardour_automix_batch.sh            #
#                                                #
#  Changes gain levels in many Ardour project    #
#      files at once.  Uses ardour_automix.sh    #
#************************************************#
for file in */*.ardour
do
ardour_automix.sh $file
done
echo " "
echo "Press ENTER to continue. "
read continue