CPU usage issues with VSTs, maybe GUI related?

(Julien "_FrnchFrgg_" RIVAUD) #21

I do not see several ways to “close” a plugin UI window that results in different behaviour (apart IIUC from the very non-obvious idea to switch to generic UI, that almost no user will think of). The only other way to “close” the window is to remove the plugin or close the session, which obviously should destroy the UI along with the DSP instances. BTW, in my home I have “light” switches that actually control a socket (the background of it I made a different color to distinguish them).

Again, the difference is about what is “closing” a window (as opposed to “quitting” for instance). When I “close” a drawer, its contents are hopefully not destroyed. I know that example is more a play on words than a real argument but you get my point: you shouldn’t assume that because you affect some connotation to a word it will be the same for others.

And no, saying to anybody they (or their decision) is stupid is not a compromise at all. Arguing that it should be changed to match other DAWs is legit (though in some cases one of Ardour’s strengths is doing differently than all other DAWs), and remember that I am not against that (I’m in fact leaning towards matching other DAWs and the handling of LV2 UIs, but my expertise is not enough for you to care about my opinion).

My experience with DAWs (not as long or thorough as yours) is that Ardour devs (and mixbus devs) are very helpful, far more than Avid (that mostly ignore anything because they can), or even back in the time the people of Logic Audio. That’s more from an user pov than a plugin writer, so YMMV. But I don’t expect Protools devs to really care about a random plugin author, esp. since there is an overwhelming number of them.

I do not think that stopping the redraw when invisible should mean that you bitblt a stale UI. Just that you actually rebuild it when going visible. It does not feel like an “horrible” hack, especially since your alternative is relying on some interpretation of the scope of “effEditor*” by DAWs that is not explicit in the spec, thus subject to change as they see fit. Of course, they have probably settled on that interpretation, not because that is the right thing to do or even because anything else is thoroughly stupid, but just because that is the interpretation that broke the fewest plugins.

(Julien "_FrnchFrgg_" RIVAUD) #22

To be clear on the technical point of view: I think that plugins UI should be destroyed when closed, mostly because they are very rarely expensive to reopen (though maybe some will complain because some plugins will be less snappy to show then).

On the other hand, I do not think that it is a good idea to refresh when not needed by an expose event. That is exactly the same as consuming lots of CPU to refresh your back-buffer 6 times more often than the number of times an expose event will paste that buffer to the actual window, just because it happens that you detected a change in some value. To me, an UI should be “display”-driven, not pushing images that nobody will ever see. vsync in graphic cards is the same idea, I think, and you can say that they tried lots of other things in their long experience.

(mike@overtonedsp) #23

Generally I find that commercial DAW developers are responsive and supportive of customers and willing to work constructively with third-party developers. When an issue is raised I don’t normally experience such push back (three years, three years, arguing about what the ‘close’ button on a window should do) - unless perhaps there is a valid technical reason I have missed, and not just “because it is”.

I think the detail you missed is that I don’t / can’t know when Ardour has made the window visible again, until after it has become visible, by which time the user has seen the stale UI. The only way to avoid this would be to update the offscreen buffer in response to UI changes, irrespective of the UI being visible, and, oh, wait, I already to that… What I don’t do is draw anything if the editor has been closed, because, it’s been closed, (not hidden).

There is the possibility that this avoids some scenario which might accidentally trigger X11’s error handler (which you categorically must not do, because it will crash the application by default, because that’s what X11 does…) But I have spelt out the preferred way to deal with this and other hosts seem to manage it.

So a meter changes and what do you do?

(That’s a rhetorical question - I know what I do, I redraw it at its new value, generate an expose event on the area which needs updating and that causes it to be relayed to the user, unless of course Ardour has hidden the window without telling me)

(Julien "_FrnchFrgg_" RIVAUD) #24

I don’t understand how this is possible: Ardour itself will not repaint your UI, it is your toolkit (that IIUC is written by you) that is responsible for repainting the UI, even if it means a copy/paste from a backbuffer. Any serious toolkit notifies of that and enables updating the backbuffer contents before it is presented to the user.

What can trigger the X11 error handler is that you access the resource after it has been freed, but that cannot happen because Ardour correctly notifies you with effEditorClose when it will do so (again, according to paul).

I think the correct behavior is to generate an invalidate request on the area corresponding to the meter, and when you get a redraw/expose event in response to that invalidate you update then present the backbuffer. That way you ensure that if the meter changes 16 times until the next time the system can redraw (e.g. because it is set to redraw only at 60fps and your meter moves fast), you only redraw once, and thus use 1/16 the cpu usage.

If your window is unmapped, you will never get the draw event that the invalidate would have triggered, and thus will not use any CPU at all for that.

Of course that doesn’t solve the fact that when a meter changes, you might need to update some internal values or compute some FFT in response (and those are not to do in the draw request, especially since you probably need all intermediate values for it to be correct). That is one of the reasons I am in favor of changing Ardour’s behavior (even if there are solutions that you could use in your code without any change of Ardour).

(mike@overtonedsp) #25

First let me say that having read your reply I think we are increasingly both saying the same thing, but just slightly differently - just to clarify:

By whatever means, the UI has been drawn by my UI toolkit into the XWindow, its there, and how it actually got there is not really relevant.
Even if I do nothing else, no other drawing whatever, that image is there, it exists, it is displayed in the window that’s it.
Therefore, the crucial point is that when the XWindow is unmapped, it and its contents will become hidden. When it is remapped, they become visible again. In this case that is the ‘stale’ version of the UI. This issue is entirely detached from any subsequent drawing my toolkit might do.

Imagine the Window as a piece of paper on which I have written something. If I unmap it (hide the paper) the paper is no longer visible, gone. When I (re)map it (reveal the paper), the paper is visible again, together with what was written on it. It does not matter what the writing says or how it got there, it remains on the paper. (and you can only know that the paper has become visible again, when it becomes visible again - I haven’t yet perfected a means to know when the user will open the or close the UI before they actually do - if I did, I could predict the lottery and retire from writing plug-ins)

I would hope that it would work without problems, as I outlined earlier in the post - I was just giving the Ardour devs a plausible reason for the behaviour other than ‘because we decided to’ - but, Ardour doesn’t correctly notify us using effEditClose - that’s the whole point.

Exactly - you’re essentially describing exactly what I do - but, the only slight difference is that when the meter value changes, it causes the meter ‘object’ to be updated with a new value and this may cause the meter object to be re-rendered to the offscreen buffer. In the case of e.g. a VU meter, this doesn’t take much CPU.
Its not really practical to redraw UI objects (controls, meters etc) to the offscreen buffer in response to expose events directly since expose events may originate from any source, including the window manager and single or multiple ‘exposed’ area(s) might span multiple partial areas of various controls and working out which one to re-render starts to get ugly very quickly. (And I would hope it would be obvious that that could be suboptimal in many cases, for example an exposed area which impinges on a small area of a very complex control).

Instead, the more elegant solution is that only when a control changes e.g. by the user dragging on it, or a meter value updating or by automation etc, that is the point at which it might get re-rendered to the offscreen buffer or flagged as needing re-rendering on the next GUI frame (but, only, if the editor is open).
Clearly if the window is hidden it cannot get user input, but, meters and FFTs etc might still be flagged as needing to be updated - because the editor is still open, its just been hidden, by Ardour - without telling the plug-in, and that is the issue we are dealing with - not just in my UI tookit, but it seems in other plug-ins too.

(Julien "_FrnchFrgg_" RIVAUD) #26

Oh, that’s not how GTK for instance behaves by default: the default expose handler just redraws the (grayish in most themes) background and that’s it. It feels like you are painting your UI into something that redraws itself automatically from the image buffer, without notifying you…

If Ardour sometimes destroys the UI without notifying with effEditorClose beforehand, I would say that is a bug and I do not think that paul would argue otherwise. I thought that what we are writing about currently is whether Ardour should do that each time the window is “hidden” with the close button, or not. I’m leaning towards “yes”, paul and robin were telling that it is currently “no” and that it was deliberate (and I view myself as not knowledgeable enough to counter their arguments in that case).

Of course, double-buffering means that even if there are several distinct expose events you only redraw once everything that has changed (updating the backbuffer), then just do what you currently do: copy from the back buffer to the screen (or rely on Xlib to do that itself if that is what it does). Of course all falls apart if Xlib doesn’t even tell you it is handling an expose event (which seems strange to me but I do not know Xlib at all).

I do not think this is a toolkit problem, since all toolkits I know enable to know whenever an expose event is incoming (and do some stuff before anything is put on the screen in response to the event). The issue is that some plugins (calf in particular as the thread implies) do not use that to decide when to update and use a busy loop or a rapid timer to redraw instead, or even redraw at each change notification (depending on what you draw, that could mean a draw at each sample which seems ridiculous).

The way I would do it (I don’t write plugins, but I did write UIs with elements that had to update in response to frequent events): when an incoming event (sample, meter change, etc.) raises the need to, flag the corresponding objects to “need update”, and invalidate the corresponding region on the plugin window.

Then wait for an expose event to collapse all these “updates” into a single one if they were close enough in time (relying on the toolkit mechanism for that). When the expose event comes, redraw all objets that “need update”, into the backbuffer, or wherever you usually paint “persistently”. Then copy the “exposed” regions from the backbuffer to the screen (this may be done automatically by your toolkit). If you want, you can skip drawing “need update” objects that have no intersection with the expose region, but keep them in the “need update” list of course.

That does not relieve you from computing some FFT coefs, but maybe you can stop doing so if you didn’t receive any expose event for more than 5s (presumably, if you get mapped again and the display starts blank it is not a problem because that would be the same if the UI is rebuilt with effEditorOpen anyway; you will have a tiny additional delay but that is it)

(mike@overtonedsp) #27

That may be GTK behaviour, I haven’t used it in a while. I deliberately simplified my explanation relating to my toolkit so as to focus on the important points, though what I’ve set out is as detailed as I can be without getting into all the low-level specifics of how my toolkit actually accomplishes managing window redrawing, and event handling (including expose and other X11 events).
I know this might make me (even more) unpopular with open source devotees, but it is a proprietary toolkit I created for my own commercial plug-in(s) which therefore limits what I am prepared to discuss about it on an open forum (I wouldn’t normally go to this much detail).

I can’t say (and don’t know) what methods the other plug-ins mentioned in the original post might do.

Essentially that’s what happens (but I can’t say if this is true of all plug-ins or the toolkits they use including those mentioned in the original post)

Fundamentally the root problem is just that Ardour does not close the window when you click the button that closes the window, that’s really the only issue.

(Paul Davis) #28

Nope. Ardour does close the window. What Ardour does not do is to destroy the window resources. We don’t send the notification because we don’t destroy the resources. So what you’re arguing for (and apparently what the behaviour of other hosts supports) is that a user operation to “close” the window ALWAYS be interpreted as a request to destroy the resources and thus allow us to notify the plugin via effEditClose.

I continue to regard this as wasteful and stupid (and in reality, VST should have all of effEditHide and effEditShow and effEditDestroy). But given that apparently so many plugin developers have relied on this design model, even if it is wasteful and stupid, it would probably be smart for us to adopt the same.

The idea that because user wants the plugin editor off the screen the only thing that can be done is to destroy/release all the resources associated with the plugin editor is just so braindead that I can hardly discuss it, but I’ve seen this pattern so many times with VST that now I just throw up my hands in the air and say “whatever”.

(Robin Gareus) #29

How does this work on Mac? When an application looses focus all child and dialog windows are hidden.

I guess Mac/Cocoa is smart enough to unset needsDisplay and not call drawRect, still a MacVST will likely consume CPU, if the visualization/analysis isn’t bound to exposure.

(mike@overtonedsp) #30

What the user is signalling by clicking the close button is that they want to close the plug-in editor (and that normally results in a window being destroyed, and together with the resources it uses - see the microsoft default implementation for WM_CLOSE etc etc).
If they just wanted it off screen, they might e.g. minimise it - which would normally result in the window being hidden, but continuing to exist, and optionally be updated, for example to keep live ‘previews’ alive - and is exactly what should happen in that case.
The same is true if for example they switch workspace, which is the other scenario I can think of in which unmapping might legitimately occur.
Its the explicit action of closing the window, which should close the window - see my example of multiple browser windows as an example.
What I’m suggesting should happen is exactly what you appear to already do for LV2, yet somehow this is seen as absolutely correct for LV2 and ‘braindead’ when applied to VST.
This is not a VST vs LV2 issue, and should not be represented as such. This is just that ardour’s current implementation of VST does not appear to have the expected behaviour, and oddly this has been characterised as an intentional design decision.

That’s a slightly different issue. If an application looses focus then I wouldn’t expect its windows to be destroyed (see switching workspaces, minimising etc) but, if the user explicitly clicks close I would expect the window to close - just that on a Mac, it might not imply quitting the application if for example you close the application main window.

(Julien "_FrnchFrgg_" RIVAUD) #31

Nobody said it was absolutely correct for LV2 as far as I read. To me it was even implied that it was an overlook.

I still think that both sides have some part to play: Ardour should probably get more in line with other hosts (just because one less difference in plugins handling means less headaches in the long run for compatibility enigmas), and plugin authors (here CALF) should try hard to not update the window too much when unneeded.

That is not the same: you’d continue to get expose events if a “preview” is running. It is the same with some compositors where your window is actually never considered “covered” by any other, because the rendering is onto some surface. In those cases, you do not get expose events just because the window is uncovered by some other (because the compositior is essentially double-buffering on top of you), but you do get expose events generated from invalidate requests (but at a reasonable rate, and only if the window is still mapped).

BTW, I have encountered exactly that kind of problem recently with my WM of choice (PekWM, I know I am strange), because it completely unmaps the window when on another workspace or minimized (instead of just hiding it or moving it offscreen or whatever), so the Xcomposite extension cannot get the window contents anymore and OBS studio didn’t work without having the window on the same workspace, but behind other windows (which works correctly) or even offscreen.

Firefox reduces the CPU used by animations when I switch workspaces, so I am pretty sure it knows it is not “mapped” anymore.

(mike@overtonedsp) #32

Dear God I swear this is going to drive me absolutely insane… For the hard of thinking, here is what should happen:

  • There are legitimate reasons to hide a window, without destroying it, this is known as ‘unmapping’ in X11. An example might be, if the user has minimised the application window, or switched workspace. This is not the same as closing it and should not be treated as such. You would not expect effEditorClose to be called for a legitimate hiding / unmapping of a window.
  • It is completely legitimate to continue to update content in a window which is unmapped (and therefore still exists but might not be visible to the user). An example might be a window has been minimised but we want the window manager to be able to show a nice ‘live preview’ on / in the taskbar or whatever.
  • Clicking the close button, that’s the ‘X’ in the corner, is an action which indicates the user wishes to close the window, intentionally destroying it, which is the default behaviour for WM_CLOSE according to the mircrosoft documentation. Its a destructive action. In this case you would expect effEditorClose to be called on the plug-in.

At present Ardour does not appear to close the plug-in window when you click the ‘X’ in the corner, but merely hides or unmaps it, therefore ‘effEditorClose’ also does not occur. You might argue that this is correct behaviour, in the sense that effEditiorClose should not be called because the window has not been closed and that makes sense - but ignores the real issue, which is that it should have been closed (correctly triggering effEditClose), because, the user explicitly closed it

(Julien "_FrnchFrgg_" RIVAUD) #33

You keep conflating “close” and "free*/“destroy” (and call stupid people who do not) whereas the Ardour authors decided that “close” would be better served by “unmap”. The VST spec is partly at fault, since it seems it calls “*close” what it describes as “destroy”. My gut feeling is that as a user I often close and reopens quickly plugin UIs (mostly because of screen estate), and thus using “destroy” as “close” is not a very good idea. Maybe “destroy after some time of being closed, like a GC” would be good.

As a developer, I think that getting back in line with all other host implementations is worthwhile, and so I agree that “close == destroy” should be done by Ardour. But again my opinion is far from authoritative in this field.

I will now step back from this discussion because I repeatedly tried to lower the tone and it nevertheless keeps getting more aggressive.

(mike@overtonedsp) #34

Lets see what happens when I hover the mouse pointer over the ‘X’ button in Cubase - which is a Steinberg application after all:

Given that a tooltip appears saying ‘close’ - I would think close is the correct term to use. And it turns out that when I click on it, effEditorClose is called in the plug-in because, the window has closed.

All I want to happen in Ardour is that when I click the close button, that’s the ‘X’ in the corner, that the window should close - as it does in Cubase, and, every other application in existence.

If I didn’t know better I’d begin to suspect this thread had descended into some farcical form of trolling. Either way, I have tried to offer the benefit of my experience (on several occasions on this forum) regrettably without success - I’m genuinely approaching it from a position of offering some knowledge to help make ardour better, both for users and third-party developers, but I don’t get the feeling the developers are receptive to this at the present time. And I despair that so much time and energy has been wasted, simply trying to communicate what the close button on a window is for. I don’t think its a constructive use of my time to contribute anything further.