I’m currently in a situation where my session is crashing, and I’m nearing the end of the project so there’s no other way for me to go forward and solve it. I’ll use this thread to keep notes on how I’m faring in this, and hopefully others can chime in with advice as well. My prospect is to bisect the plugins until I find the offending ones. I’m using a paper notebook to track which plugins I disabled. It could be that one instance of the same plugin is crashing and the other isn’t. I’m going on a channel-by-channel basis rather than type of plugin.
I had partial success with the following procedure to load a crashing session without having to bisect anything. It worked for me until this morning.
Load my session in safe mode
Session → Close (but don’t close Ardour!)
Load my session in normal mode
Ardour says there are VST3 plugins missing. The plugin manager window pops up. I click the button to discover new and updated plugins.
Session → Close (but don’t close Ardour!)
Load my session in normal mode
So I’d need to load my session 3 times, once in safe mode, once in normal mode but with missing plugins, and once with functional plugins. Unfortunately this morning this no longer works, and so I need to actually bisect all the plugins. The task is nothing short of onerous, wish me luck.
I ran Ardour from command line, and my session loaded fine. What an anticlimax. I’m not sure if I should be happy or annoyed. I finished my project! I did.
I remember reading an article about why Ardour doesn’t sandbox plugins. Whatever the reasons were, and whether I agree with them or not, doesn’t matter. I have a tingling feeling I’ll have another crashing session soon enough. A few months back I regularly crashed Ardour by writing a few lines of innocent Lua. That was when I was looking for a way to A/B two busses.
I see two outstanding issues for me, to follow up later in this thread:
The first issue, why did the session load? Did something change in the environment? Or was is something about the plugins registry? Maybe it’s not a bad plugin itself, perhaps it’s something about how Ardour loads plugins. In most cases Ardour doesn’t crash when I work with plugins, it crashes when I load the session.
The second issue, I’m thinking the hunt for a problematic plugin should be at least partly automatable. For example, listing plugins, dividing them into halves, keeping track of which plugin was tested with what result, that’s all a job for a script, not for a human. I don’t know yet what options scripting gives me and what level of control I have. I can list plugins from Lua (there already is a plugin for that). Scripting enabling/disabling plugins might be hard, perhaps it could be done from luasession? Otherwise I could write code to edit XML to enable/disable plugins in the session file. Then what’s left, is a bit of code that will find out if loading the session succeeded or not. How that would look like, I’m not sure, perhaps run Ardour from command line, wait a bit (some sessions take a little time to load), and see if it’s still running.
I’ll keep updating the thread if/when I make progress on this.
My session started crashing again, but now I’ve delivered the project so I can calmly analyze what’s going on. Let’s start with the most obvious thing:
What about those disappearing VST3 plugins?
The symptom I’m seeing:
Open a session, add a plugin, work on the session
Close the session, exit Ardour
Try to open the session, Ardour crashes
Open the session in safe mode, session opens
Close the session without exiting Ardour
Open the session in normal mode
Ardour shows a dialog that it can’t find two VST3 plugins!
Is it possible that the crash is related to these two specific VST3 plugins? Does Ardour have functionality in which it disables plugins that misbehave? How to understand what’s happening?
The issue returned today. I’m still trying to figure out what’s the best sequence of steps to recover the session reliably. It looks like the problem revolves around two plugins from AudioThing that I’m using in my session: Blindfold EQ and Fog Convolver 2, both are vst3 plugins.
I noticed an additional symptom: when Ardour loads my session, sometimes after finishing the loading it spontaneously (without me pressing play in the session) plays some very loud and very distorted sounds. I’ve now created a habit of lowering the monitoring volume, because the sound is usually very very loud.
I can’t reproduce it on command, but symptoms come and go. Is there any way for me to capture diagnostics when it happens? (which I don’t control)
Yes, they are two plugins from AudioThing, Blindfold EQ and Fog Convolver. I’m using plugins from 3 sources only: inbuilt Ardour plugins, LSP, and AudioThing. AFAIK AudioThing uses JUCE for plugins.
The issue reappeared, in a different project. The usual sequence is:
I start Ardour, choose the session, click “open”
Ardour shows the starting screen and shows all the loading messages, until it flashes very briefly “loaded history” (or something like that), and the Ardour window disappears - that’s how I discover the problem returned. Now, I’m going on the slightly unpredictable recovery journey.
I start Ardour again, click Safe Mode, open my session. The session loads.
I press CTRL+S and close the session (but not Ardour)
The sessions list dialog reappears, I uncheck Safe Mode and select my session again. Now the session loads, but a bunch of plugins appear missing. - but note that they are still installed and are perfectly fine! Nothing changed there.
I open the Plugin Manager window (or maybe it opens automatically?) and I re-scan plugins (“new and updated”)
I close the session (but not Ardour) again.
I open the session with Safe Mode unchecked again.
After step 8 there are two possibilities.
One is that the session now works (for a time, at least)
One is Ardour crashes again. In this case I grit my teeth, and try to work in the Safe Mode, with all the plugins disabled. For tracking, that’s fine. For mixing, that’s kind of annoying, I just try to find tasks that don’t require plugins - like editing some details or balancing levels. After some time, just try to open the session with Safe Mode unchecked, and eventually it works.
Having background in software development, I know I’ll eventually get my session to work again. It’s a bit worse when it happens in front of a client. I already have the skill to stoically call a five minute break (without saying why), or say that I need a few minutes “to resolve a tech issue”, but if it keeps happening I’ll need to look for ways to reduce the frequency with which this happens, or I need a reliable workaround. The current workaround is just too stochastic.
I have a feeling that it’s an integration boundary problem. Ardour people would say, “oh, a plugin caused the crash”, and the plugin makers would say “I don’t see anything wrong with my plugin, it works on Reaper!” Meanwhile, I need both to play nice with each other.
What’s the advice here, what should I do?
Bisecting plugins doesn’t work, because the whole process is not deterministic. Alternating between Safe Mode and normal mode “fixes” the issue sometimes.
If you start Ardour from a terminal window you should get some sort of error message when it crashes.
If you post those here maybe someone can decipher what’s causing the crash.
The reason it can’t find your VSTs after a crash is possibly because it corrupted the files or folder that Ardour uses to keep track of VST plugins
In Linux those files are in ~/.cache/ardour8/vst (for Ardour v8) so once you have a working session you should take a backup of that folder, to have something to compare with after the crash.
You can also use the backup to replace the supposedly corrupt cache files with the working ones, to save you from rescanning the plugins.
@automaciej Sadly the core file you posted is on IRC only the first step. That file is specific to your system. See Generating Backtraces from Core Dumps the debugging Ardour page:
Ardour9 --gdb
(gdb) core-file "ardour9-crash-at-session-start.core"
(gdb) thread apply all bt
They are all VST3 plugins, most from AudioThing and a few from ACMT. Here’s an example message:
I know that AudioThing uses JUCE, and I don’t know what ACMT uses. I’m now wondering, one hypothesis would be some form of disagreement between Ardour and JUCE?
- but note that they are still installed and are perfectly fine! Nothing changed there.