Hello! I don’t know if this has been discussed or not but it could be potentially helpful for devs. https://drone.io/
Ardour is not “deployable”, and there is essentially no automated testing mechanism. For reference, Digidesign’s automated testing of ProTools was a separate. major software development effort and I’ve heard that a full test took 1 week to run.
drone.io appears to be targetting software that provides services (e.g. via the net) or that has an automated testing framework available.
paul, i wonder if some sort of automated regression testing might be worth the effort, particular since it might be a job for us users, rather than devs. like, we might design a session with some standard characteristics, wire up some test signal generators via jack_connect, fire a few OSC commands at ardour, make sure the basics are always there. then the expected outcome of the audio tracks could be verified via null tests, and the state of the session file interpreted with some XSLT magic. might that make sense? otoh, ardour git HEAD has been remarkably robust in the “easy-to-test” department, and all major bughunts i’ve witnessed on IRC were about far subtler issues, so maybe it’s not the most pressing thing atm…
I wish there was an effective way to automate testing - without the burden of implementing and maintaining it outweighing any benefit - which I think can be summed up as:
‘The time you save by getting a computer to do something for you is normally spent watching it to make sure it doesn’t do it wrong’
Even for relatively uncomplicated plug-ins, I find testing takes a significant part of the release cycle, and I can’t imagine its possible to manually test every aspect of something as complex as Ardour (and still leave any time for development)
Large software companies often employ specific software test engineers or departments. Software bugs are frustrating for developers and especially for users, but for a ‘free’ project, which is technically still in beta, I think the deal is partly that as a user you are also the software test department.
Thanks for the explanation Paul.