Some times when exporting midi files, by playing them back from timidity I get the message
take2.mid: Illigal Variable-length quantity format.
Although timidity plays such files without problem, in analysing them from other sources (e.g. Python) the tracks can’t be handled because they are “corrupted” as my program reports.
Does any one know what Illigal Variable-length quantity format means and how should I avoid it?
Many thanks
Amir
Probably best would be to post a MIDI file which generates that error so someone else can try to analyze where the problem exists in the file. The error message is too vague to guess which quantity is noted as incorrect.
Sorry for long not answering! I have attached a midi file to this repository, which seems to be faulty!
When I play back this file with timidity Take17_General\ MIDI\ Synth-1.3.mid I get
Playing Take17_General MIDI Synth-1.3.mid
MIDI file: Take17_General MIDI Synth-1.3.mid
Format: 0 Tracks: 1 Divisions: 19200
Take17_General MIDI Synth-1.3.mid: Illigal Variable-length quantity format.
Playing time: ~6 seconds
Notes cut: 0
Notes lost totally: 0
I would very much like to know what this Illigal Variable-length quantity format. is appearing. This leads to problems, for instance when analyzing the file with a programming-lang. For instance trying to read the midi file with Python (pretty_midi.PrettyMIDI(midipath)) a Value Error like this appears:
ValueError: MIDI file has a largest tick of 144115188075998575, it is likely corrupt
I suspect Timidity cannot deal with our MIDI files. You should likely export it instead, to avoid Ardour-specific (but 100% legal) stuff is not included in the file.
Thank you for your reply! I have exported the the file from Ardour, by right clicking on the MIDI region and choosing Export... option. Or did you mean something different with export? How could I avoid Ardour-specific stuff and get a legal midi export?
This is what mf2txt (MIDI file to text converter) shows as the contents of the example file. Does this look basically correct?
MFile 0 1 19200
MTrk
0 SeqSpec 99 01 ea 5b
6530 On ch=1 n=68 v=123
6530 SeqSpec 99 01 ea 5c
9010 On ch=1 n=72 v=104
9010 SeqSpec 99 01 ea 5d
10390 Off ch=1 n=68 v=64
10390 SeqSpec 99 01 ea 5e
13140 Off ch=1 n=72 v=64
13140 SeqSpec 99 01 ea 5f
21960 On ch=1 n=78 v=120
21960 SeqSpec 99 01 ea 60
24990 On ch=1 n=74 v=107
24990 SeqSpec 99 01 ea 61
26100 Off ch=1 n=78 v=64
26100 SeqSpec 99 01 ea 62
27780 Off ch=1 n=74 v=64
27780 SeqSpec 99 01 ea 63
36300 On ch=1 n=64 v=102
36300 SeqSpec 99 01 ea 64
40160 Off ch=1 n=64 v=64
40160 SeqSpec 99 01 ea 65
51180 On ch=1 n=76 v=93
51180 SeqSpec 99 01 ea 66
54490 Off ch=1 n=76 v=64
54490 SeqSpec 99 01 ea 67
70740 On ch=1 n=68 v=70
70740 SeqSpec 99 01 ea 68
74600 On ch=1 n=72 v=103
74600 SeqSpec 99 01 ea 69
75150 Off ch=1 n=68 v=64
75150 SeqSpec 99 01 ea 6a
77080 On ch=1 n=80 v=95
77080 SeqSpec 99 01 ea 6b
77630 Off ch=1 n=72 v=64
77630 SeqSpec 99 01 ea 6c
80110 On ch=1 n=82 v=107
80110 SeqSpec 99 01 ea 6d
81220 Off ch=1 n=80 v=64
81220 SeqSpec 99 01 ea 6e
83420 Off ch=1 n=82 v=64
144115188075935068 Meta TrkEnd
TrkEnd
I do see a suspiciously large value like you pointed out before, in this case 144115188075935068.
@paul is the 19200 at the beginning of the file a correct representation of 19200 ticks per quarter note? I’m kind of skimming the SMF spec I found as I go, that seems like the right field for tick quantization. It seems really high, but Ardour has that supermega clock or whatever is used as the underlying quantum to join music and audio time, so I guess could be right.
Is that final value the delta time stamp at track end time? Because even at 19200 ticks per quarter that is something like 62 billion minutes at 120bpm. Doesn’t seem reasonable.
This is the raw file data for anyone interested:
00000000: 4d54 6864 0000 0006 0000 0001 4b00 4d54 MThd........K.MT
00000010: 726b 0000 0110 00ff 7f04 9901 ea5b b302 rk...........[..
00000020: 9044 7b00 ff7f 0499 01ea 5c93 3090 4868 .D{.......\.0.Hh
00000030: 00ff 7f04 9901 ea5d 8a64 8044 4000 ff7f .......].d.D@...
00000040: 0499 01ea 5e95 3e80 4840 00ff 7f04 9901 ....^.>.H@......
00000050: ea5f c474 904e 7800 ff7f 0499 01ea 6097 ._.t.Nx.......`.
00000060: 5690 4a6b 00ff 7f04 9901 ea61 8856 804e V.Jk.......a.V.N
00000070: 4000 ff7f 0499 01ea 628d 1080 4a40 00ff @.......b...J@..
00000080: 7f04 9901 ea63 c248 9040 6600 ff7f 0499 .....c.H.@f.....
00000090: 01ea 649e 1480 4040 00ff 7f04 9901 ea65 ..d...@@.......e
000000a0: d60c 904c 5d00 ff7f 0499 01ea 6699 6e80 ...L].......f.n.
000000b0: 4c40 00ff 7f04 9901 ea67 fe7a 9044 4600 L@.......g.z.DF.
000000c0: ff7f 0499 01ea 689e 1490 4867 00ff 7f04 ......h...Hg....
000000d0: 9901 ea69 8426 8044 4000 ff7f 0499 01ea ...i.&.D@.......
000000e0: 6a8f 0a90 505f 00ff 7f04 9901 ea6b 8426 j...P_.......k.&
000000f0: 8048 4000 ff7f 0499 01ea 6c93 3090 526b .H@.......l.0.Rk
00000100: 00ff 7f04 9901 ea6d 8856 8050 4000 ff7f .......m.V.P@...
00000110: 0499 01ea 6e91 1880 5240 81ff ffff ffff ....n...R@......
00000120: ffdf 00ff 2f00
Yes, in German-european space ca 62 Milliard 
I have found a (non-optimal) way to discard this very big tick-number for my usage (setting the tick-limit to a very high number), but this has the risk of out-of-memory problems! 
What exactly is the Meta TrkEnd line, is it possible to get rid of it without practical problems?
It is required in the file, but should have a realistic value. It should be the end point of the file in ticks.
You could try setting the value to some value after the last note event manually to check how timidity opens the file with just that single change.
I was hoping to see confirmation that 19200 is a realistic tick value for ardour. I am used to seeing 192 on older gear, so it isn’t clear if ardour actually uses 100x tick precision, or if that is a value which ardour uses internally but should not be in an export file.
I found that 19200 is a legal value for tick per quarter in a MIDI file. It is on the high side, 960 was a value I saw quoted for a couple of other DAW, but definitely legal, so should not be any cause for a program to flag the file as invalid.
I tried to duplicate this in a test session in Ardour 9.8 (ardour.org Windows build).
I added a MIDI track with GM Synth and recorded a few beats of MIDI data.
I then right-clicked on the MIDI region, selected the region name, then selected export from the menu and picked a file name and location. The file played correctly in a couple of applications I tried which did flag the file from linked repository as having an error.
So currently it is not clear what is different on the OP system that causes this issue (which if I read correctly is intermittent since it is described as occurring “some” times, not every time).
This is the contents of the file from my export, showing a reasonable value for the track end:
MFile 0 1 19200
MTrk
0 SeqSpec 99 01 8c 1e
150660 On ch=1 n=60 v=108
150660 SeqSpec 99 01 8c 1f
171670 On ch=1 n=64 v=111
171670 SeqSpec 99 01 8c 20
173670 Off ch=1 n=60 v=64
173670 SeqSpec 99 01 8c 21
190510 On ch=1 n=62 v=113
190510 SeqSpec 99 01 8c 22
190670 Off ch=1 n=64 v=64
190670 SeqSpec 99 01 8c 23
210740 On ch=1 n=65 v=115
210740 SeqSpec 99 01 8c 24
212580 Off ch=1 n=62 v=64
212580 SeqSpec 99 01 8c 25
230600 On ch=1 n=64 v=113
230600 SeqSpec 99 01 8c 26
233060 Off ch=1 n=65 v=64
233060 SeqSpec 99 01 8c 27
249860 On ch=1 n=67 v=102
249860 SeqSpec 99 01 8c 28
251060 Off ch=1 n=64 v=64
251060 SeqSpec 99 01 8c 29
267880 On ch=1 n=65 v=113
267880 SeqSpec 99 01 8c 2a
269870 Off ch=1 n=67 v=64
269870 SeqSpec 99 01 8c 2b
285870 On ch=1 n=62 v=110
285870 SeqSpec 99 01 8c 2c
287460 Off ch=1 n=65 v=64
287460 SeqSpec 99 01 8c 2d
303840 On ch=1 n=60 v=106
303840 SeqSpec 99 01 8c 2e
304240 Off ch=1 n=62 v=64
304240 SeqSpec 99 01 8c 2f
386860 Off ch=1 n=60 v=64
458550 Meta TrkEnd
TrkEnd
Raw file data for comparison:
00000000: 4d54 6864 0000 0006 0000 0001 4b00 4d54 MThd........K.MT
00000010: 726b 0000 00f8 00ff 7f04 9901 8c1e 8999 rk..............
00000020: 0490 3c6c 00ff 7f04 9901 8c1f 81a4 1290 ..<l............
00000030: 406f 00ff 7f04 9901 8c20 8f50 803c 4000 @o....... .P.<@.
00000040: ff7f 0499 018c 2181 8348 903e 7100 ff7f ......!..H.>q...
00000050: 0499 018c 2281 2080 4040 00ff 7f04 9901 ....". .@@......
00000060: 8c23 819c 6690 4173 00ff 7f04 9901 8c24 .#..f.As.......$
00000070: 8e30 803e 4000 ff7f 0499 018c 2581 8c64 .0.>@.......%..d
00000080: 9040 7100 ff7f 0499 018c 2693 1c80 4140 .@q.......&...A@
00000090: 00ff 7f04 9901 8c27 8183 2090 4366 00ff .......'.. .Cf..
000000a0: 7f04 9901 8c28 8930 8040 4000 ff7f 0499 .....(.0.@@.....
000000b0: 018c 2981 8334 9041 7100 ff7f 0499 018c ..)..4.Aq.......
000000c0: 2a8f 4680 4340 00ff 7f04 9901 8c2b fd00 *.F.C@.......+..
000000d0: 903e 6e00 ff7f 0499 018c 2c8c 3680 4140 .>n.......,.6.A@
000000e0: 00ff 7f04 9901 8c2d ff7c 903c 6a00 ff7f .......-.|.<j...
000000f0: 0499 018c 2e83 1080 3e40 00ff 7f04 9901 ........>@......
00000100: 8c2f 8585 3c80 3c40 84b0 0aff 2f00 ./..<.<@..../
FYI: yesterday I committed some changes so that Ardour no longer writes “SeqSpec” metadata events into MIDI files created for export.
I didn’t find any complaints from applications about the SeqSpec events.
Although it does look odd that the delta time values for the SeqSpec events are exactly the same as the preceding Note On or Note Off value. Since it is a delta time, if the SeqSpec occurs at the same time as the note event shouldn’t the delta be 0?
I’m pretty sure that the 144e^15 end value in the original file is what is causing the problem, that exceeds the maximum value allowed for a variable length value. Assuming that midi2text parsed it correctly, I have not hand decoded yet to make sure that is correct.
I could not duplicate the behavior on my attempt at exporting a region, so not sure what could trigger that.
Yeah, it just seemed wrong to put them in an exported SMF since those events are only for Ardour. The delta time thing … maybe I’ll look into it. Within Ardour, we don’t care about the time, but it does seem incorrect on the face of it.
I was not able to duplicate the behavior using the same method, so anything you notice about your session that might be triggering the behavior would be useful to report.
A program I have found used to convert MIDI file data into readable text is this program:
https://pieter.vanoostrum.org/en/midi2text-mf2t-t2mf/
If you have multiple MIDI files which have this problem you could use the mf2t utility to convert them to text and verify if the problem is always that the end delta time stamp is a ridiculously large value, or if the problem moves around to other items as well.
I tested some other (corrupted) exported midi files, what I see in the output of mf2t is at the end always that huge number similar to the example you have pasted above. BTW the way I have been using for exporting the midi files (and I don’t know other ways to export, if you do please let me know to test it also that way) right click on the region → Export
I am on a Lubuntu Laptop, using Ardour 9.7.0
Using the Lubuntu build, or the ardour.org build?
I had tried the same procedure using 9.8 from ardour.org when I could not duplicate.
ardour.org build.
Strange, maybe try to play some very fast runs e.g. Arpeggios, Glissandos on the keyboard (on a midi-keyboard connected to Ardour), this is what I have been doing facing the huge number at the end.
Ok, I did install the 9.8 still no improvements in the exported midi.
I still am getting the huge number at the end, every now and then, I can’t tell when this happens, but it pops up some times leading to the corrupted files, example:
MFile 0 1 19200
MTrk
0 SeqSpec 99 01 e1 2c
14070 On ch=1 n=69 v=107
14070 SeqSpec 99 01 e1 2d
51520 Off ch=1 n=69 v=64
51520 SeqSpec 99 01 e1 2e
51520 On ch=1 n=63 v=117
51520 SeqSpec 99 01 e1 2f
58690 Off ch=1 n=63 v=64
58690 SeqSpec 99 01 e1 30
62820 On ch=1 n=56 v=115
62820 SeqSpec 99 01 e1 31
97530 Off ch=1 n=56 v=64
144115188075951866 Meta TrkEnd
TrkEnd
I thought it might have to do with the overlapping of the notes, but no! I used the MIDI > Remove overlap to ensure there is no sound overlap. I removed the last line (with the ugly number), remade it to a midi file with t2mf, and the problem was gone! 