Export feature ides

[peetaCodes (Peter Cox)]

I’ve been using Ardour as my only DAW for more than three years now, and I really think it’s a fantastic piece of software and I love the direction development is going. (9.7 and 9.8 updates in particular were incredible).
During this time period, I though about some quality of life improvements to Ardour’s export feature:

1. The ability to set the timespan to export via location markers.
   Currently Ardour allows for exporting one or multiple ranges of the session’s timeline delimited by a range marker, which is great and really comes in handy!
   During my usage of Ardour though I realised it would be useful to export a range of the session timeline delimited by location markers: for example if I want to export part “A” of the session, and I have location markers “Part A” and “Part B” already in place, it would be convinient to export from marker “Part A” to marker “Part B” without having to create a new range marker “Part A” just for this one time, not to mention the fact the name overlapping.

2. The ability to chose the soundwave format displayed on the exported analysis report (if it’s going to be exported). currently the interractive window allows you to visualise the exported file’s audio waves in any of the three scales: no scale (default), “logscale” or “rectified”; but the exported file always uses the default (no scale). It would be convinient if this was toggable before export. I think it would be best to apply the same scale to every report in the case mutiple files are being exported in one go (I’m not sure though). Also it should be possible to export multiple scales at once (so produce “…_logscale.png” “…_rectified.png”), just like you can export multiple ranges at once.

3. Analysis report overwriting protection: currently if a user exports a session into two different formats in one go with the same filename (and in the same folder), like “my-session.wav” and “my-session.flac”, as the filename for the report is calculated to be the corrispondent filename with the extention replaced with".png", both of the reports will have filename “my-session.png”, so after the export one overwrites the other. It would be nice to have a fallback that detects this and renames the report’s filenames to something like “my-session_wav.png” and “my-session_flac.png” (perhaps preserving the audio files’ filenames).

4. A feature for MIDI export: currently Ardour supports MIDI stem export, that exports every MIDI track in an Ardour session to its own MIDI file. It would be nice to include the option to export every MIDI track into a single “type 1” (also called multi-track or multi-source) MIDI file, where each MIDI track on the session is its own instrument (track) in the output MIDI file.
   This would be very handy to generate sheets music: let’s say I have made a complex ochestral arrangement in Ardour using MIDI tracks, whith this feautre I could create a single MIDI file with the arrangement for any instrument that could be directly imported in programs like MuseScore to immediately generate an exaustive sheet music.
   I had already made a Lua script that does this (I’m going to be honest: I used a lot of help from chatGPT: I had never worked with Lua before. I’m sorry), though it has the only issue of disalligning regions on the same track (so every track needs to have exactly one or no regiorns otherwise they will overlap/severe timing issues will occur. Regions from the same track overlap only with regions from that same track). Other than that I’d say it’s mostly OK.
   Here is the script in its entirety in case you’re wondering (I’m pretty sure it’s unresonably long but I wan’t able to make it any shorter or all sorts of timing issues would pop up in the output file):

local PPQ = 1920 -- Ardour's ticks per quarter in your session

local NON_DRUM_CHANNELS = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15 }

local function safe_isnil(obj)
  if obj == nil then
    return true
  end
  local ok, ret = pcall(function()
    return obj:isnil()
  end)
  if not ok then
    return false
  end
  return ret and true or false
end

local function clamp7(v)
  v = math.floor(tonumber(v) or 0)
  if v < 0 then return 0 end
  if v > 127 then return 127 end
  return v
end

local function clamp_channel(v)
  v = math.floor(tonumber(v) or 0)
  if v < 0 then return 0 end
  if v > 15 then
    v = v % 16
  end
  return v
end

local function sanitize_filename(s)
  s = tostring(s or "session")
  s = s:gsub("[%z\1-\31/\\:%*%?\"<>|]", "_")
  s = s:gsub("%s+", "_")
  s = s:gsub("_+", "_")
  s = s:gsub("^_+", ""):gsub("_+$", "")
  if s == "" then
    s = "session"
  end
  return s
end

local function u16(n)
  n = math.floor(tonumber(n) or 0) & 0xFFFF
  return string.char((n >> 8) & 0xFF, n & 0xFF)
end

local function u32(n)
  n = math.floor(tonumber(n) or 0) & 0xFFFFFFFF
  return string.char(
    (n >> 24) & 0xFF,
    (n >> 16) & 0xFF,
    (n >> 8) & 0xFF,
    n & 0xFF
  )
end

local function vlq(n)
  n = math.floor(tonumber(n) or 0)
  if n < 0 then n = 0 end
  local bytes = { n & 0x7F }
  n = n >> 7
  while n > 0 do
    table.insert(bytes, 1, (n & 0x7F) | 0x80)
    n = n >> 7
  end
  return string.char(table.unpack(bytes))
end

local function meta_text(meta_type, text)
  text = tostring(text or "")
  return string.char(0xFF, meta_type) .. vlq(#text) .. text
end

local function safe_to_ticks(beats_obj)
  if not beats_obj then
    return 0
  end
  local ok, ticks = pcall(function()
    return beats_obj:to_ticks()
  end)
  if not ok or ticks == nil then
    return 0
  end
  ticks = math.floor(tonumber(ticks) or 0)
  if ticks < 0 then
    ticks = 0
  end
  return ticks
end

local function region_start_ticks(region)
  local ok, ticks = pcall(function()
    return region:position():beats():to_ticks()
  end)
  if not ok or ticks == nil then
    return 0
  end
  ticks = math.floor(tonumber(ticks) or 0)
  if ticks < 0 then
    ticks = 0
  end
  return ticks
end

local function lower_name(s)
  return tostring(s or ""):lower()
end

local function is_drum_track(track_name)
  local n = lower_name(track_name)
  return n:find("drum", 1, true)
      or n:find("perc", 1, true)
      or n:find("kit", 1, true)
      or n:find("timp", 1, true)
end

local function infer_program(track_name)
  local n = lower_name(track_name)

  if n:find("piano", 1, true) or n:find("keys", 1, true) or n:find("keyboard", 1, true) then
    return 0
  end
  if n:find("epiano", 1, true) or n:find("electric piano", 1, true) or n:find("rhodes", 1, true) or n:find("wurlitzer", 1, true) then
    return 4
  end
  if n:find("organ", 1, true) then
    return 16
  end
  if n:find("acoustic guitar", 1, true) or n:find("nylon guitar", 1, true) then
    return 24
  end
  if n:find("guitar", 1, true) then
    return 27
  end
  if n:find("bass", 1, true) then
    return 33
  end
  if n:find("violin", 1, true) then
    return 40
  end
  if n:find("viola", 1, true) then
    return 41
  end
  if n:find("cello", 1, true) then
    return 42
  end
  if n:find("contrabass", 1, true) or n:find("double bass", 1, true) then
    return 43
  end
  if n:find("harp", 1, true) then
    return 46
  end
  if n:find("strings", 1, true) or n:find("string", 1, true) then
    return 48
  end
  if n:find("voice", 1, true) or n:find("choir", 1, true) or n:find("vocal", 1, true) then
    return 52
  end
  if n:find("trumpet", 1, true) then
    return 56
  end
  if n:find("trombone", 1, true) then
    return 57
  end
  if n:find("horn", 1, true) then
    return 60
  end
  if n:find("sax", 1, true) then
    return 64
  end
  if n:find("oboe", 1, true) then
    return 68
  end
  if n:find("clarinet", 1, true) then
    return 71
  end
  if n:find("flute", 1, true) then
    return 73
  end
  if n:find("lead", 1, true) then
    return 80
  end
  if n:find("pad", 1, true) then
    return 88
  end

  return 0
end

local function allocate_channel(track_index, track_name)
  if is_drum_track(track_name) then
    return 9
  end
  local pool_index = ((track_index - 1) % #NON_DRUM_CHANNELS) + 1
  return NON_DRUM_CHANNELS[pool_index]
end

local function midi_on_velocity(v)
  v = tonumber(v)
  if v == nil then
    return 64
  end
  if v > 0 and v < 1 then
    v = v * 127
  end
  v = math.floor(v + 0.5)
  if v < 1 then
    v = 64
  end
  if v > 127 then
    v = 127
  end
  return v
end

local function midi_off_velocity(v)
  v = tonumber(v) or 0
  if v > 0 and v < 1 then
    v = v * 127
  end
  v = math.floor(v + 0.5)
  if v < 0 then
    v = 0
  end
  if v > 127 then
    v = 127
  end
  return v
end

local function tempo_meta_from_bpm(bpm)
  bpm = tonumber(bpm) or 120
  if bpm <= 0 then bpm = 120 end
  local mpqn = math.floor((60000000 / bpm) + 0.5)
  if mpqn < 1 then mpqn = 1 end
  if mpqn > 0xFFFFFF then mpqn = 0xFFFFFF end
  return string.char(
    0xFF, 0x51, 0x03,
    (mpqn >> 16) & 0xFF,
    (mpqn >> 8) & 0xFF,
    mpqn & 0xFF
  )
end

local function timesig_meta_from_meter(num, den)
  num = math.floor(tonumber(num) or 4)
  den = math.floor(tonumber(den) or 4)
  if num < 1 then num = 4 end
  if den < 1 then den = 4 end

  local dd = 0
  local d = den
  while d > 1 do
    d = d // 2
    dd = dd + 1
  end

  return string.char(0xFF, 0x58, 0x04, num & 0xFF, dd & 0xFF, 24, 8)
end

local function collect_midi_track_events(track, assigned_channel)
  local events = {}
  local track_max_tick = 0

  local track_name = "MIDI Track"
  local ok_name, name = pcall(function()
    return track:name()
  end)
  if ok_name and name and name ~= "" then
    track_name = name
  end

  local program = infer_program(track_name)
  local channel = clamp_channel(assigned_channel)

  local ok_pl, playlist = pcall(function()
    return track:playlist()
  end)
  if not ok_pl or safe_isnil(playlist) then
    return track_name, "", 0, channel, program, 0
  end

  local ok_mpl, mplaylist = pcall(function()
    return playlist:to_midiplaylist()
  end)
  if not ok_mpl or safe_isnil(mplaylist) then
    return track_name, "", 0, channel, program, 0
  end

  local ok_rl, rlist = pcall(function()
    return mplaylist:region_list()
  end)

  local regions = {}
  if ok_rl and not safe_isnil(rlist) then
    local ok_tbl, tbl = pcall(function()
      return rlist:table()
    end)
    if ok_tbl and type(tbl) == "table" then
      regions = tbl
    end
  end

  table.sort(regions, function(a, b)
    local ap = region_start_ticks(a)
    local bp = region_start_ticks(b)
    if ap == bp then
      return tostring(a) < tostring(b)
    end
    return ap < bp
  end)

  for _, region in ipairs(regions) do
    local ok_mr, mr = pcall(function()
      return region:to_midiregion()
    end)

    if ok_mr and not safe_isnil(mr) then
      local ok_model, model = pcall(function()
        return mr:model()
      end)

      if ok_model and not safe_isnil(model) then
        local region_offset = region_start_ticks(region)

        local ok_notes, note_list = pcall(function()
          return ARDOUR.LuaAPI.note_list(model)
        end)

        if ok_notes and not safe_isnil(note_list) then
          local ok_ntbl, notes = pcall(function()
            return note_list:table()
          end)

          if ok_ntbl and type(notes) == "table" then
            for _, note in ipairs(notes) do
              local ok_nt, start_beats = pcall(function()
                return note:time()
              end)
              local ok_ln, len_beats = pcall(function()
                return note:length()
              end)
              local ok_nn, nn = pcall(function()
                return note:note()
              end)
              local ok_vel, vel = pcall(function()
                return note:velocity()
              end)
              local ok_off, off_vel = pcall(function()
                return note:off_velocity()
              end)

              if ok_nt and ok_ln and ok_nn and ok_vel and ok_off then
                local start_tick = region_offset + safe_to_ticks(start_beats)
                local dur_tick = math.max(1, safe_to_ticks(len_beats))
                local end_tick = start_tick + dur_tick

                if end_tick > track_max_tick then
                  track_max_tick = end_tick
                end

                nn = clamp7(nn)
                vel = midi_on_velocity(vel)
                off_vel = midi_off_velocity(off_vel)

                events[#events + 1] = {
                  tick = end_tick,
                  order = 1,
                  data = string.char(0x80 | channel, nn, off_vel)
                }
                events[#events + 1] = {
                  tick = start_tick,
                  order = 2,
                  data = string.char(0x90 | channel, nn, vel)
                }
              end
            end
          end
        end
      end
    end
  end

  table.sort(events, function(a, b)
    if a.tick == b.tick then
      return a.order < b.order
    end
    return a.tick < b.tick
  end)

  local body = {}
  body[#body + 1] = vlq(0) .. meta_text(0x03, track_name)
  body[#body + 1] = vlq(0) .. meta_text(0x04, track_name)
  body[#body + 1] = vlq(0) .. string.char(0xC0 | channel, program)

  local last_tick = 0
  for _, ev in ipairs(events) do
    local tick = math.max(0, math.floor(tonumber(ev.tick) or 0))
    local delta = tick - last_tick
    if delta < 0 then delta = 0 end
    body[#body + 1] = vlq(delta) .. ev.data
    last_tick = tick
  end

  body[#body + 1] = vlq(0) .. string.char(0xFF, 0x2F, 0x00)

  return track_name, table.concat(body), #events, channel, program, track_max_tick
end

local function build_conductor_track(max_tick)
  local events = {}
  local tm = nil

  pcall(function()
    tm = Temporal.TempoMap.read()
  end)

  if tm and not safe_isnil(tm) then
    local step = math.max(1, math.floor(tonumber(Temporal.ticks_per_beat) or PPQ))
    local last_bpm, last_num, last_den = nil, nil, nil

    local tick = 0
    while tick <= max_tick do
      local tp = Temporal.timepos_t.from_ticks(tick)

      local bpm = 120
      local ok_bpm, bpm_val = pcall(function()
        return tm:quarters_per_minute_at(tp)
      end)
      if ok_bpm and bpm_val ~= nil then
        bpm = tonumber(bpm_val) or 120
      end

      local num, den = 4, 4
      local ok_meter, meter = pcall(function()
        return tm:meter_at(tp)
      end)
      if ok_meter and meter and not safe_isnil(meter) then
        local ok_num, n = pcall(function()
          return meter:divisions_per_bar()
        end)
        local ok_den, d = pcall(function()
          return meter:note_value()
        end)
        if ok_num and n ~= nil then
          num = math.floor(tonumber(n) or 4)
        end
        if ok_den and d ~= nil then
          den = math.floor(tonumber(d) or 4)
        end
      end

      if last_num == nil or num ~= last_num or den ~= last_den then
        events[#events + 1] = {
          tick = tick,
          order = 1,
          data = timesig_meta_from_meter(num, den)
        }
        last_num, last_den = num, den
      end

      if last_bpm == nil or math.abs(bpm - last_bpm) > 0.0001 then
        events[#events + 1] = {
          tick = tick,
          order = 2,
          data = tempo_meta_from_bpm(bpm)
        }
        last_bpm = bpm
      end

      tick = tick + step
    end
  end

  if #events == 0 then
    events[#events + 1] = { tick = 0, order = 1, data = timesig_meta_from_meter(4, 4) }
    events[#events + 1] = { tick = 0, order = 2, data = tempo_meta_from_bpm(120) }
  end

  table.sort(events, function(a, b)
    if a.tick == b.tick then
      return a.order < b.order
    end
    return a.tick < b.tick
  end)

  local body = {}
  body[#body + 1] = vlq(0) .. meta_text(0x03, "Conductor")

  local last_tick = 0
  for _, ev in ipairs(events) do
    local delta = ev.tick - last_tick
    if delta < 0 then delta = 0 end
    body[#body + 1] = vlq(delta) .. ev.data
    last_tick = ev.tick
  end

  body[#body + 1] = vlq(0) .. string.char(0xFF, 0x2F, 0x00)
  return table.concat(body)
end

local function session_export_path()
  local base_dir = Session:path()
  if not base_dir or base_dir == "" then
    base_dir = ARDOUR.user_cache_directory(-1)
  end

  local session_name = "session"
  local ok_name, sname = pcall(function()
    return Session:name()
  end)
  if ok_name and sname and sname ~= "" then
    session_name = sanitize_filename(sname)
  end

  return ARDOUR.LuaAPI.build_filename(base_dir, session_name .. "_midi_multitrack_musescore.mid")
end

function factory(params)
  return function ()
    local routes = Session:get_tracks()
    if safe_isnil(routes) then
      print("Ardour MIDI export: no tracks found.")
      return
    end

    local ok_table, route_table = pcall(function()
      return routes:table()
    end)
    if not ok_table or type(route_table) ~= "table" then
      print("Ardour MIDI export: unable to read track list.")
      return
    end

    local midi_tracks = {}
    local warned_channel_wrap = false
    local song_max_tick = 0

    for _, route in ipairs(route_table) do
      local dtype = ""
      local ok_dt = pcall(function()
        dtype = route:data_type():to_string():lower()
      end)

      if ok_dt and dtype:find("midi", 1, true) then
        local ok_track, track = pcall(function()
          return route:to_track()
        end)

        if ok_track and not safe_isnil(track) then
          local track_index = #midi_tracks + 1
          local provisional_name = "MIDI Track"
          local ok_name, name = pcall(function()
            return track:name()
          end)
          if ok_name and name and name ~= "" then
            provisional_name = name
          end

          local assigned_channel = allocate_channel(track_index, provisional_name)
          if track_index > #NON_DRUM_CHANNELS and assigned_channel ~= 9 and not warned_channel_wrap then
            print("Ardour MIDI export: more than 15 non-drum MIDI tracks; MIDI channels will wrap.")
            warned_channel_wrap = true
          end

          local track_name, track_chunk, event_count, channel, program, track_max_tick =
            collect_midi_track_events(track, assigned_channel)

          if track_max_tick > song_max_tick then
            song_max_tick = track_max_tick
          end

          midi_tracks[#midi_tracks + 1] = {
            name = track_name,
            chunk = track_chunk,
            events = event_count,
            channel = channel,
            program = program
          }
        end
      end
    end

    if #midi_tracks == 0 then
      print("Ardour MIDI export: no MIDI tracks found.")
      return
    end

    local conductor = build_conductor_track(song_max_tick + (PPQ * 4))
    local out_file = session_export_path()

    local fh, err = io.open(out_file, "wb")
    if not fh then
      error("Ardour MIDI export: cannot open output file: " .. tostring(err))
    end

    fh:write("MThd", u32(6), u16(1), u16(#midi_tracks + 1), u16(PPQ))
    fh:write("MTrk", u32(#conductor), conductor)

    for _, tr in ipairs(midi_tracks) do
      fh:write("MTrk", u32(#tr.chunk), tr.chunk)
    end

    fh:close()

    print(string.format(
      "Ardour MIDI export: wrote %d MIDI track(s) to %s",
      #midi_tracks,
      out_file
    ))
  end
end

[x42 (Robin Gareus)]

Good ideas.

(1) Is already possible if you use section (aka. arrangement) markers. Those automatically extend to the next marker. Since 9.7 they also show up as export ranges. You can right-click and change existing markers into arrangement markers. Make sure to also shoe the Arrangement Ruler (in Menu > View > Rulers, or right-click on the ruler-header).

(2) Since 9.8.3 Preferences > Appearance > Waveform settings are now used as defaults.

(3) is definitely a buglette.

Edit: Note that if you only pick different file formats then the report for both is identical. This is why only one .png is created. – But it is currently possible to export to e.g. “peak normalized .flac” and “loudness normalized .wav” at the same time, which would require different files.

(4) Ardour uses SMF directly, so you could just grab the files from the session’s interchange/ folder, but that has similar issues as you mentioned:

@peetaCodes

it has the only issue of disalligning regions on the same track (so every track needs to have exactly one or no regions otherwise they will overlap/severe timing issues will occur.

This is why there is only stem-export (and single region export from the context menu). Ardour allows for overlapping and layered regions which may or may not be transparent. There is no trivial way to decide what should be exported. So I dare say for the time being your nice script will likely be the best option.

[peetaCodes (Peter Cox)]

Oh, I see. I didn’t know versions 9.7 and 9.8 already included ideas 1 and 2.

As for idea 4, I’m afraid I didn’t explain the problem of “overlapping regions” properly:
in my test case, in the Ardour session no region was overlapping. Each one started either exactly or after the previous one had ended. The problem was that my script didn’t handle region chnages well: the exported file was spotless untill we reach a bar (or beat) were in the original session a “region change” happened: suddently the notes in the MIDI file for that single track weren’t synced at all, timings were completely off and notes from further points in time were sometimes overlapping with others that came before (as a result of the timings being off).

Apparently the way I calculated timings was in some way region-specific, but since it was used globally for the whole file as soon as a region changed, the timings completely broke. (That’s why I fixed it using only one region per track). I still have no real clue why that hapeened.

@x42

Ardour uses SMF directly, so you could just grab the files from the session’s interchange/ folder,

This actually worked, but my goal was to have only one MIDI file containing every track, and those files only store each region singularly. You see, software like MuseScore wouldn’t let me “append” a MIDI file into an existing score: it’s always one score per MIDI file, that’s why I wanted to generate a single multi-track MIDI file containing every track and every region. Anyways, if that’s still unrealistic for the time being, I have no problem continuing to use the Lua script.

[x42 (Robin Gareus)]

@peetaCodes

Oh, I see. I didn’t know versions 9.7 and 9.8 already included ideas 1 and 2.

(2) is not yet in a released version. I only added that 35 mins ago (Use logscale/rectified preferences for export report · Ardour/ardour@18cbbd5 · GitHub) - It will be in tomorrow’s https://nightly.ardour.org/ build Ardour 9.8.3 or later.

[peetaCodes (Peter Cox)]

@x42

(2) is not yet in a released version. I only added that 35 mins ago

Wow I dind’t know I timed this post this precisely

[x42 (Robin Gareus)]

@peetaCodes

Wow I dind’t know I timed this post this precisely

I added it in response to your post. It was a quick and easy fix.