Ardour development and AI

I think that if AI is going to be used in Ardour, its main task should be to point out potential solutions for long standing, hard to find bugs that are causing stability issues. Some of its findings may be very insightful, and hopefully easy to implement .

A larger project for AI would be to move the GUI away from YTK/GTK2. What should it move to ? Java based GUI ? GTK3? GTK4? I don’t think AI would do a very good job of that in its current state.

Ardour is an excellent DAW. The features and the UI are some of the best available. No need to vibe-code any new features.

I think stability would help Ardour / Mixbus in the long run.

1 Like

Your original argument was “anything AI”, not “generative AI”. So why are you still arguing?

P.S. Inpainting is generative AI these days. Valid usecase.

Today, I tried comparing a local, open-weights model (GPT-OSS) with Claude with the same task: make a simple 3D game engine, written in Rust, with free look and procedurally generated textures, and generate a small demo level to walk around in.

Claude took my prompt and had a working demo up and running in about 10 minutes which met all my requirements. The code quality, at a glance, appeared to be very good.

GPT-OSS took about 45 minutes before it had a code base that wouldn’t compile and politely informed me that it tried, but the task was too complex for it. After this, I tried again, however, focusing on one subsystem at a time; basic code layout, rendering, game loop, input/output. After a lot of back-and-forth across multiple conversations, I had something that wasn’t complete, but a usable start.

Some things I realised about this:

  • Frontier and local LLMs are both far away from hitting their plateau. I really thought a year ago that both would not improve any more than they had. A year ago, all I could get any local model to do was hallucinate left and right before going into a spiral that would resemble a nervous breakdown were it written by a human. They are clearly still improving, and quickly.
  • The ethical dilemmas about environmental impacts may be temporary - the local coder ran at 1/10th of the speed of the cloud model, but it’s still early days, and all the power it consumed came from the solar panels on my roof. Hardware continues to improve and could eventually eat the energy cost of LLMs on brute force alone, even if the LLMs themselves never got more efficient.
  • The copyright considerations are nuanced. In my attempt with the local LLM, what I came away with was not just a pile of code but also a long audit trail showing a clear proof of work and authorial intent on my part - all the code it wrote followed guidance from my (poor) judgement and decision-making, so it’s not a case of “monkey took the photo so it can’t be copyrighted”, it’s far more murky than that. OTOH, I certainly could not have written any of the code myself in a reasonable amount of time.
  • If I had absolutely no clue how a game engine was composed, I’d have gotten nowhere using the local model. And if I need to know anything new about how to do something in a game engine, the first thing I’ll look to is exactly the same as an LLM would be looking to: prior art. So I’m not too worried about accidental plagiarism when it comes to code. If it’s ok for a human to do it slow, I think it’s ok for an LLM to do it fast.

This all leads me to think that the Ardour team’s approach here is probably the best attitude to have. The tech is getting good enough now that code cannot be dismissed as ‘slop’ anymore out-of-hand, and over time, it will become more and more accessible and the environmental concerns will lessen. At this point, if the AI bubble pops and all the datacentres turn into paintball arenas, we are left with local models that can do a good enough job that you can still get more done with them than without them.

Ultimately the health of the project and quality of the product is what matters most. I think if you can show you clearly had a significant presence in the judgement and decision-making that resulted in any new code appearing in Ardour, then the risk that the code will turn out to be incompatible with the GPL is greatly lessened. Banning LLM-generated code from people outside the core team further helps insulate against that risk. And ultimately if it’s the Ardour team that drives its development and not a bunch of LLMs, and can show their work in the code and collateral around it, then it’s all good.

2 Likes

Do it, use LLMs for the grunt implementation. You remain the architect, developer, and source of knowledge, reason and control. You won’t be doing any less work, the work will be different, higher level, more planning - probably less enjoyable than coding - but progress will be faster because an LLM can work on multiple issues at once.

2 Likes

I am afraid this is a wishful thinking, for sure not what we are seeing now.

As an example, Berlin is planning to double its electricity supply infrastructure until 2030 (that is in four years). I have just looked up the statistics: in Germany (the biggest electricity consumer in Europe) the consume is basically the same since 1990 (on a side note no, neither the mobile phones nor the Internet did make a dent in it, let alone it being “one of the most energy-hungry technologies in human history” lol).

I doubt that heat pumps and electric vehicles alone could be seen as the reason for such a gigantic need. I also hear that due to the sheer number of requests, licences to build new data centres in the Frankfurt area are now deferred to at least the year 2035. Maybe I am wrong, but I fear there is a very direct connection.

This is very much the problem: destroy now, think tomorrow.

Unfortunately, unlike the woods and forests being wiped out to make place for them, paintball arenas do not produce oxygen.

3 Likes

I think reduction in energy use will come primarily from newer AI systems that are more efficient. For example the work that Taalas has been doing (search for Chat Jimmy) and also the next generation of AI that will move on from LLMs and transformers - whatever that may be.

1 Like

So I guess we will just have to wait until those newer systems are functional, before taking an informed decision about chatbot use in Ardour code, right? :wink:

1 Like

What would we gain by doing that?

Unfortunately, reliable X11 is being dropped. On my system, Xwayland seems to cause severe bugs with Ardour. My thinking was GTK3 / GTK4 or something else may be more compatible with Wayland. I should have mentioned Wayland compatibility instead. Wayland has been slightly faster than X11 from what I can see.

EDIT: yes there are x11 alternatives, but setup and configuration on new distributions may be a headache for some.

It is interesting in the light of the original post here that the X11, quartz (macOS) and win32 backends for GDK2 are all about 16K LOC …

3 Likes

For me AI as advanced helper tool, small pieces of code checked by experienced developers are reasonable things.

2 Likes

Ask it to create best elstique time algortythm in the whole world, then read it, understand it completely, rewrite the damn thing your own way and GPL it :slight_smile: . (I’m kidding, but it would be great if this turns out to be possible…).
Ardour isn’t exactly lacking features(only some)…It lacks more out of the box music-creation friendliness. Think FL, LIVE, CUBASE - there’s many sounds and instruments available right away. And there’s all this work around Ardour you have to pull off to have a full music workstation, which isn’t realy Ardour’s fault. You have to configure your linux distro properly, you have to find/install/check/roadtest plugins you need, you have to find sounds and check they’re licenses etc…you simply don’t have to do that with something like FL.
One feature i would give AI to create is transforming midi track to musical notation (could be useful).
Another one is completely separate workspace tab to transform it to something akin FL, to base Ardour’s workflow immediatley on patterns with view containing different instruments/sounds.
So if these features break, no biggy.
But i wouldn’t do anything that could make big problems if it breaks/disapears in future releases. No need to stress that i’d completely separate AI generated code from the rest of it.

If AI can handle it, it would be a good use of it. As it is now, AI ignores certain prompts and still does what it wants. But in a few years or months, would it be capable of that with less intervention?

That grunt work of fixing the GUI would be a beneficial use.

Does the GUI GTK2/YTK cause latency in the UI currently? I know on my system it may be a bit of a bottleneck. The audio engine runs well, but the gui starts to lag. And Ibthink there was an issue with the Mac GUI as well.

Paul has explained many times before that the main editor UI does not use any GTK2 components.
GTK2 is used for a few selection dialog box elements.

Hi Paul and Robin,

Thanks for the open and realistic opening post. My short answer to your thoughts is: proceed with caution and consideration – pick only the best raisins from the AI cake.

The old saying applies here perfectly: fire is a good servant, but a bad master.

Another extremely important perspective that shouldn’t be forgotten is the environmental impact and hype around AI. This trend definitely needs to be watched critically. Here in Finland, we already have a unfortunately concrete example of this: 300 hectares of forest were cut down for a Google data center project before official permits were even approved, and now local water bodies in the area are at risk. The energy and water consumption as well as land use required by the AI boom are not just theoretical concerns; they leave severe scars on the environment faster than regulations can keep up.

In summary:

Use AI strictly as a limited “servant” to remove severe development bottlenecks that human resources wouldn’t otherwise cover.

Keep humans as the masters and architects.

Maintain a critical and responsible attitude toward the broader environmental impacts of the technology.

Good luck with the decision-making!

12 Likes

I absolutely see where you’re coming from and the discussion is not that different down here in Australia. Honestly though, I don’t think that the vast majority of these proposed AI datacentre projects will actually happen. You may have read about Firmus’ cancelled IPO here. The market is still yet to sober up and my hope is that we’ll see more and more stories like theirs.

I truly hope so.
In the meanwhile see the post before yours :frowning:

“A pessimist is just an optimist with experience.”

6 Likes

So tell me…
How many LOC are needed to re-implement smooth-scrolling? :slight_smile:

:thinking:

Last time I checked: 7 files changed, 83 insertions(+), 19 deletions(-), without Claude. That’s without all the potentiel side-effects & polishing of course.

3 Likes

Is there a chance this will see the light of day? AFAIR, there is no consensus among the devs on whether it should be implemented at all.

Nevertheless, I sincerely hope that we’ll see the Enable manually programmed smooth scrolling checkbox in the preferences soon. :slight_smile:

2 Likes