Is Open Source a diversion from what users really want?

When I started working on Ardour, it never occured to me to do anything other than use the GNU Public License (GPL), the most well-known way to release “open source” software. At that time, it was a choice driven by a combination of:

  • my passionate belief in what is more appropriately called "free/libre software"
  • an awareness that I'd probably need help developing Ardour. The open source model seemedto me the best way to make it possible for others to contribute (no matter what their motivations might have been).
  • the desirability of being able to use dozens of software libraries released under GPL-related licenses

Of course, developing software with complexity on the level of Ardour’s is never going to be easy, and finding other people willing and able to contribute to such a project is always going to be hard, whether you’re an open source project or a proprietary company.

However, underlying both of those reasons why I wanted to use the GPL was a conviction the access to the source code was critical to both:

  1. giving users the freedom they deserved
  2. attracting developers (or even just "power users") to contribute to the project.

I remain convinced that access to source code is a fundamental part of the "four freedoms" that Richard Stallman has outlined as the basis of the concept of "free/libre software". But as described at great length and exhaustive detail by Berlin-based electronic musician and developer Louigi Verona, it's not quite that simple.

Meanwhile, how could anyone really contribute to the project in any substantive way without source code access? If they were going to add functionality, or extend it or in some other way modify it, source code access seems like a basic and absolute requirement.

A recent thread on our forums has made me revisit these assumptions, and this has led me to have some doubts about what "open source" really means for a project like Ardour.

Forum member Musocity wants to be able to extend Ardour without having to build the program from source. If you read the thread, you'll see both myself and co-developer Robin Gareus pushing back on this concept several times. Nevertheless, Musocity continues to use Reaper as a counter-example in which much greater levels of user-driven "extensions" are possible without any access to the source code and without any requirement to rebuild the program.

My gut reaction continues to be something along the lines of

Are you kidding me? We give you full access to everything in Ardour, not just some pre-selected functionality exposed via a scripting language. You can build it on almost any platform, add/remove/enhance almost anything you can imagine ... and you keep pushing for a 2nd-rate scripting interface just so that you can do stuff without dealing with the build process?

But this forum thread has made me keep returning to two points I mentioned above. Specifically in the form of follow up questions:

  1. are users truly being given freedom by confronting them with a technological infrastructure that almost none of them can comprehend?
  2. does the requirement to rebuild the program, or from a different perspective, to write C++ code, attract or deter developers to/from the project?

These are not easy questions to answer.

Let's start by pointing out what Ardour already offers: a very sustantial Lua API providing access to the majority of the program, and with it the ability to write both DSP code and higher-level functionality, all without rebuilding Ardour or dealing with C++. This has all been Robin Gareus' effort, and he has done an amazing job (aided by just how suitable Lua is for this sort of thing - partly why Reaper uses it too, no doubt). What is missing is the ability to construct arbitrary graphic user interface components from Lua. This puts distinct limits on what can be done with scripting in Ardour, even given how much is already possible.

Reaper stands as an existence proof of what can be done when the scripting capabilities are essentially all-encompassing. Ableton Live, with both Max4Live and other "scripting systems" offers slightly less extensive capabilities, but still somewhat more than Ardour in terms of GUI integration.

Nevertheless, it remains the case that nobody except for Reaper's (or Live's) developers can modify fundamental aspects of the program. The work that we have been doing on Ardour 6.0 would never have been possible to do via Lua, and that would be true in the context of Reaper or Live as well. So the first thing that we need to note is:

  1. the scripting interface for a DAW can vary in terms of what it makes possible to accomplish.
  2. this is particularly true in terms of integration into the "main body" of DAW's own GUI.
  3. no matter what the scripting interface offers, it does not allow anyone to do fundamental work on the internals of the DAW. A DAW that cannot do cue monitoring will never become one that can because of a script extension. The same goes for full latency compensation, misdesigned region/clip lists and an almost inexhaustible list of other features that cannot be implemented via script extension system.

Musocity, it appears, doesn't really care about any of this: they've seen what you can do with Reaper's scripting interface, and it seems entirely reasonable to them that the same ought to be true in Ardour whether or not the entire program's source code is available.

Which brings us to the second aspect of why this is complicated. Even 20 years ago there were full time web developers. There were also so-called "application developers" who typically worked entirely inside database-connected development tools to create ways to view and edit data. In the two decades since Ardour started, we've seen an entire generation of people whose job description includes "programming" but who have never (or almost never) compiled a piece of software in their life. The web development infrastructure that has grown up over the last two decades has seen huge numbers of people creating software in ways that never require them to take "source code" and transform it into "a program". They write "scripts" (be it in Javascript, Java, Python, Ruby or whatever they prefer) and the necessary magic happens to ensure that what they've written actually executes, somehow. Even for the most sophisticated web development stacks, where there's some notion of "build systems" and "deployment", these only have a superficial resemblance to the workflow involved with a desktop application written in a compiled language.

In my limited experience interacting with people who develop on a web stack, the build process for a program like Ardour is frequently a massive stumbling block to any participation they might have considered. They may not mind dealing with poorly documented (or undocumented) APIs, complex data structures and mind-boggling program control flow. But tell them that after they make a change, they have to "build" the program and that in some circumstances this will take several minutes to complete ... enthusiasm starts dying rapidly. And now consider what happens when these developers are on platforms (primarily Windows and macOS) where they cannot issue a single command (e.g. apt-get build-dep ardour) to set up their build environment, but must painstakingly build/install 2GB of source code-provided 3rd party libraries, before they can even build Ardour for the first time.

It's not entirely surprising that a project like this doesn't have many active developers at any given point in time!

Of course, there are other factors too: really getting into Ardour development means being comfortable with (in no particular order): real time programming, parallel/thread programming, cross-platform development, C++ idioms, model-view-controller design, the GTK+ toolkit, some level of DSP knowledge, a non-trivial understanding of audio and MIDI hardware, the MIDI protocol, and lots more. Even if Ardour was paying more developers, it would be extremely hard to find people with the right background and outlook.

When I released Ardour under the GPL, my vision was that by virtue of it being an open source project (technically orthogonal to its status as "free/libre software"), it would be possible, even encouraged, for other developers to participate and get involved in extending its capabilities. Musocity's forum thread, and their insistence that "all this should be possible by scripting" has made me wonder if this belief was ever true, and in particular if it is still true.

Why isn't the Reaper model better? Technically-inclined users can do insane things with a script, and in so doing can easily address most of the things that particular users want the program to do. Almost no Reaper user cares that they cannot build Reaper from source, cannot modify the fundamentals of the program, cannot redistribute a modified Reaper to their colleagues/friends. It matters much more to them that someone outside of the Reaper team can cook up a script that can do "just about anything". That's what freedom looks like to Reaper users (or so it appears), and giving them the source to Reaper would barely change that, if at all.

Verona touches on so many aspects of this in his piece. The demographics/background of computer users in 2020 is so very, very different from the way things were when Stallman began the concept of "free/libre software". Back then, "freedom to tinker" really did mean the freedom to read and edit source code, and to rebuild programs from source. Today, even though this is still a foundational aspect of the concept of "free/libre software", the freedom that many users want doesn't come from source code access at all. It comes from applications that enable their users to easily customize things to "about the level that most users care about".

Nevertheless, the concept of free/libre software is still vitally important to me and millions of other people. As mentioned above, even the best script extension system (or any form of program customizability) cannot replace source code (and build system) access in terms of providing the kind of freedom to tinker (and thus freedom to learn) that Stallman (and many others) envisaged.

But perhaps for applications like Ardour, ones that do not yet exist, there ought to be a different development pathway. I remember once wondering if we should have implemented the entire GUI in PyGTK (i.e. Python). We didn't, and most of my curiosity was about whether it would have helped or hindered our development process. However, had we done so, one of the consequences would have been that many changes to the program would have been made simpler, easier to access and would require no "rebuild".

I wonder if going forward, large-scale apps like Ardour ought to (as Reaper did relatively early in its life) consider the "script extension system" to be a vital and critical part of the application infrastructure. This would mean, for example, writing large parts of "core functionality" using this system, rather than dropping back into C++ to get things done. There are precedents for this: GNU Emacs, for example, is at some level written in C, but almost everything about the program is actually constructed in Emacs Lisp, its own "scripting extension". The C core of Emacs is so small and so irrelevant that it almost doesn't matter that it is there: if you want to modify or extend Emacs, you (almost always) write Lisp, not C.

Forcing the "core" developers to, as the saying goes, "eat the same shit" as regular users forces them to focus their attention on the quality of said "shit". Removing the need to rebuild the application after most changes opens the application to contributions from people who cannot deal with (1) the idea of compilation and/or (2) the reality of compilation.

Would we have attracted more developers over the years if Ardour had been more accessible to programmers with skills in Python, Javascript or Ruby? It's hard to know. I have no idea how many people (absolute or as a fraction of the user base) have written notable extensions for Reaper (or Live). It's possible that it would make no difference whatsoever, and would merely divert developer time away from one level (C++) where we can function efficiently and happily and divert it to another ("scripting extension") that doesn't actually enable much at all.

I don't know that answers to any of the questions I've mentioned above. I do know that Robin did an amazing job bringing an incredible level of scripting with Lua into Ardour, and that the things you can't do with it are very much a result of our joint intention - the intention that people who want to modify or extend Ardour should plan on working on the (open) source code for the program, not by convincing us to expand to scope of our scripting support.

But perhaps that's wrong. What do you think?

25 Likes

The very fact that you are thinking deeply about these things and putting your thoughts out on the forum is really fantastic.

I don’t know the answer myself. I can say that I developed an affinity to Reaper because I was able to add some classical features (single-window source/destination editing, for example) that made my job a lot easier. I could not have done this without scripting. Essentially I was able to think logically and put something into motion without worrying about learning a language. At 39, I’m probably never going to pick up a language at this point and will rely on Bash scripting etc (however inefficient I may be!) and be happy enough.

There’s enough I love about Ardour that I’ve moved entirely over to it (and Linux) for everything audio-related. My three recent additions to the issue tracker are about all I can come up with for feature requests! As long as you were able to maintain your vision for Ardour at the same time as allowing for more customization and/or extensions, I see no issues, personally.

Correct me if I’m wrong but LUA seems a little bit of a steeper climb than Reaper’s scripting? Also, for Reaper, SWS extensions are what give a whole new level of power for scripting. Perhaps something similar for Ardour would allow both ideologies to flourish?

4 Likes

I think this is about the 90% / 10% rule. However you would redesign Ardour you would only meet 90% of the users needs, there is no way to satisfy the whole 100% and no program in the world does that, including reaper.

It is very important that Ardour is released under GPL. If the developers were some day unable to continue their work for some reason the license guarantees that someone can take over the project. If reaper would land in financial trouble users might lose the product and all their “investment” in it.

I’m 54 and I think its never too late to learn programming. I have in the recent years learnt Python and Go and last year I learnt C and C++. I’m thinking about diving into the Ardour codebase after my C++ skills solidifies somewhat. The availability of the code allows me to learn from it and tinker with it.

Web development is a completely different beast because there the browser is the platform. The browser platform is here to stay but it will not replace the os, these will continue to live side by side. C and C++ are still in the top 5 of the most popular programming languages (number 2 and 4 February 2020) (https://www.tiobe.com/tiobe-index/) so I don’t think there is any lack of people skilled in these crafts. I also don’t see any need for Ardour to becoming a new programming platform in itself.

One thing that could make it easier for people to tinker with Ardour code would be easy access to all required library versions. A virtual machine os image with all the dependencies already installed would be ideal, but that would be more work for developers. Although the virtual os would not need to be up to date only the build environment itself.

My requirements for a DAW are quite small and I am one of the few whos needs Ardour as it is satisfies for 100%. Also i’ve been exploring source code for some open source projects and found out that C and C++ skills are essential for a coder living in the Linux Open Source ecosystem. Most of the projects use those languages so if you want to collaborate you need to understand these. So from a Open Source programmers point of view one still needs C / C++ skills had Ardour a global scripting interface or not.

4 Likes

I can see both sides. Being comfortable with “low-level” programming and compiling myself, I tend to look for the “easy way out” first and try to fix it from the “outside”, preferably a script.

It is true that being able to script something quickly is a good tool for trying something out and get a quick feel for if it’s working or not (rapid prototyping). I think this is especially true for user interfaces.

I think scripting can enable more users to hack something together, have it tested and actively refined further by the community, to maybe then be converted to C++ where it can run faster, enable more functionality, etc.

I guess my conclusion is that ardour scripting should be as extensive as possible and also encourage users to submit scripts somewhere. But every script should be treated as a potential feature request for the lower level code.

2 Likes

I don’t care for open source but I care about freedom. I found Ardour because I was looking for a free alternative to Reaper. I settled on Ardour because of its scripting capabilities, which thanks to VAMP plugins meet some of my needs that Reaper can’t.

The myth about “open source” is that you give source code access and developers will suddenly appear and help out. This is true for a tiny number (probably <1%) of projects. Most projects have a single developer. So I think Ardour is doing pretty well in developer numbers.

Almost no Reaper user cares that they cannot build Reaper from source, cannot modify the fundamentals of the program,

Most users are… users. So it’s true but unsurprising that only a small percentage of Reaper users care about access to its source code - this is true for all software with a significant user base. But just because the percentage overall is small it doesn’t mean that the number who do care is insignificant. A lot of those who develop scripts for Reaper would love access to its full source. For GNU/Linux users one important part of Reaper’s source is “open” - libSwell. I believe it’s been through user contribution to this GUI library that it’s now possible to run Reaper natively on GNU/Linux. There are also C++ based extensions such as SWS and ReaPack which have been built by the users.

The average user of a DAW is not a programmer. But many are capable of writing little macros/scripts to do simple tasks. The Lua bindings provided by Ardour are perfect for this and you should continue to add more as appropriate. I agree with you that larger things like GUI systems really should be part of the main codebase.

Then there are some users (I consider myself one) who are capable of programming but are not familiar with C/C++. We are able to build the project from source, make minor tweaks/changes, and help debug issues. But we’re not able to contribute significanly to the codebase due to our lower skill set.

And of course you have users that are also skilled C/C++ developers who are able to make changes to the codebase and have a much smaller need for a macro/scripting system than an average user, but can still benefit from it.

So I don’t think “open source” is a diversion from what users want, although (as with all software) it’s probably not important to most users (and neither is lua scripting). You have different users with different skills and different needs, and you cater for them all beautifully.

While the Reaper devs have left the scripting system open ended (and in some ways messy) you have kept Ardour’s scripting very focused. It is there to provide a specific functionality and it does it well. The Lua bindings that are already available give us lots of options. So many in fact that I am already using more scripts than the GUI can handle neatly. I have resorted to running most of them from the script editor.

5 Likes

It’s not a diversion, it’s a necessary fundamental layer on top of which more user-focused things like the ones you mentioned ought to be built.

3 Likes

Please don’t change anything :slight_smile:

Imho, a scripting interface should be an optional feature, and not an integral part of the product. If it can be extended to allow more control, that’s nice but moving responsibilities to scripts so that a wider audience can contribute to the core product doesn’t make it a better product. You would probably be reviewing lots of low-quality code and discussing why this shouldn’t be merged, instead of focusing on improving the product yourself. A community-driven repository of Lua scripts could be an optional asset.

The linked text of Louigi Verona also mentions the huge amount of available audio plugins for Linux, but the percentage of good quality ones is rather low. (By the way, it was a dead link for me). (https://medium.com/@louigi.verona/linux-audio-an-overview-13823cc32c42)

Here’s a (harsh) quote from Linus Torvalds:

‘C++ is a horrible language. It’s made more horrible by the fact that a lot
of substandard programmers use it… Quite frankly, even if the choice of C were to do nothing but keep the C++ programmers out, that in itself would be a huge reason to use C.’ (http://harmful.cat-v.org/software/c++/linus)

As he sees C as the language of choice for low-level Kernel code, I would see modern C++
potentially mixed with C as the language(s) of choice for a GUI DAW application.

Ardour is a great success and I think the development model doesn’t need to be changed at all. I guess there’s just not enough Linux C/C++ developers with interest in Audio who still have some spare time in this world.

It’s also crucial that Ardour stays GPL’ed. What about pushing Harrison towards GPL? :slight_smile:

7 Likes

Back when I was a major contributor to Mixbus we had several enquiries from MSVC programmers wanting to get involved in the Windows build. But in every case, what put them off was the enormous investment required in building the support libraries. In fact apart from me, I don’t think there was even a single programmer who made it as far as building Mixbus itself (they all gave up, long before reaching that stage…) It’s definitely an issue that (from a newbie’s perspective) having to build support libs is very much the boring bit. I’m not sure what the answer is but maybe more programmers would be attracted if that could be addressed somehow?

GPL and source availability might have been one of the main reasons why Ardour has come so far. The git source repo from 2005 to 2020 lists 104 people who have contributed to the project (only first 100 listed here: https://github.com/Ardour/ardour/graphs/contributors). When counting the number of code lines they added / removed I wonder where Ardour would be without those contributions.

I hope you guys are not considering relicensing Ardour and moving away from GPL.

You ask “is open source a diversion from what users really want” and imho the answer is that users just want cheap / free software, but open source is the mechanism that makes it possible to develop good software with few human resources. I’m a firm believer in the GPL and I think that without it the Linux ecosystem probably would have never happened.

6 Likes

I suppose I should stress that the discussion I was trying to start is absolutely NOT about relicensing or moving from the GPL.

It’s about the contrasting styles of “freedom” offered by (1) GPL’ed source code access (2) “complete” scripting capabilities (using Reaper as one example).

9 Likes

Hmm this is a tough one, in part because there are multiple different reasons why open source is critical, moreso than Richard Stallman identifies in his freedoms I believe. I am trying to formulate my thoughts on this, but frankly it would take hours to type out a good response I think, so it may not happen given my schedule right now.

Suffice to say I think the Louigi Verona is probably on to something from reading his intro, though I have not read the entire paper. I think there are people that utilize the software because it is free cost first and foremost, but still want to be able to customize it, and don’t mind giving back, or some people that truly see a need and want to give back. But as others have mentioned there is a significant time investment, not in the learning to code so much, as learning the software on which you are coding. I am probably more advanced than most users in that side of things with Ardour, but would still reach for a good scripting environment first for many tasks, rather than jumping in the code for Ardour because of the significant time investment needed for me to understand the ‘how’ and ‘why’ of Ardour’s code, and how to navigate it. Heck I only ever really worked on the control surface code a long time ago with John and even that took some time to wrap my head around how it worked, and has been completely rewritten (For the better) since then, so that is not a wasted investment:)

A different way of looking at it could be ‘what needs to be low level’ vs ‘what could be handled by scripting’. I don’t think the answer is to make everything that can be scripted, a script. But I don’t t hink that just because it can be handled by the source doesn’t mean it is best handled by the source.

Anyways I am trying to think of how to best phrase a slightly less rambling version of this and more detailed. If I get a chance I may come back later tonight and try again to type it out.

Scripts are inherently open-source (although not necessarily free software).
The topic title is somewhat odd, since the source-code of Rea Scripts is readable.

As for scripting in Ardour, long term I’d like to build the complete GUI using Lua as glue, while widgets remain C/C++ (and the engine obviously has to remain in C/C++, maybe we’ll throw in some rust in places, e.g. ctrl surfaces come to mind). This is mainly because In many cases the overhead of using C++ for the GUI is significant and Lua scripts are lightweight and flexible.

As opposed to Reaper, Ardour does not have a stable script interface, but libardour methods are exposed as-is. The main goal was not to allow 3rd party to extend Ardour, but open the possibilities.

Since Ardour is free-software to begin with, we do not need scripting to allow 3rd parties to extend Ardour. While for Reaper et al, scripts are the only way to extend the software.

5 Likes

As an example at how accessible Reaper’s scripting is:


Essentially you search for actions on the left panel and add them onto the right panel, re-ordering by dragging as you desire. The script itself might look meaningless but it achieves a simple classical editing function known as 4-point editing with two sets of in and out points. You can see that the third action from the bottom (on right-hand side) calls another of my scripts to create a short crossfade. I just had to think through everything logically. Obviously the fact that Reaper+SWS made all these functions accessible in plain language was huge.

That isn’t Reaper scripting. That’s calling the scripts via an action macro.

This is scripting inside Reaper:

And this is scripting in Ardour:

Reaper’s scripting interface is a little more sophisticated than Ardour’s (but it’s ugly). I do most of my scripting in CudaText anyway and just use the script editor in Ardour to test and debug.

2 Likes

I stand corrected! Yes, I’m referring to custom actions which would also be very powerful for Ardour users (and definitely more accessible!). I call them scripts because like BASH, I’m taking pre-existing commands/actions and chaining them together to make something useful. There’s no way I’ll be attempting Reascript proper!

You can chain Ardour commands together quite easily with a lua script. It’s essentially the same as Reaper’s actions but you have to do it in text, a GUI version would definitely be handy.

Here’s one I made to skip to and play the next region.

  Session:request_transport_speed(0.0, true) -- stop playback
  Editor:access_action ("Editor", "select-topmost") -- select first track
  Editor:access_action ("Editor", "select-next-route") -- select second track
  Editor:access_action ("Editor", "playhead-to-next-region-sync") -- move cursor
  Session:request_transport_speed(1.0, true) -- start playback
2 Likes

A GUI version would, indeed, be handy. For regular folk wanting to use this function they would only need to see the following:

Stop playback
Select first track
select next (second) track
move cursor to next region
start playback

Editor:access_action ("Editor", "select-topmost") isn’t exactly user-friendly. No offense intended at all to how this has been set up. Just a perspective about what I consider accessible for my skill level. As has been discussed already, those with C/C++ can contribute at that deep level. Those with LUA knowledge have that option already (ideally with no restrictions placed on LUA scripts). The question remains as to whether there is another sub-set of users who would enjoy/benefit from a surface-level macro/custom action toolbox. I’d say, a resounding “yes”!

I was surprised to not see the words “Blender” anywhere on this page or in the comments? While I love spending my day in Emacs I think Blender is a much better direct model for Ardour.

https://en.wikibooks.org/wiki/Blender_3D:_Noob_to_Pro/Advanced_Tutorials/Python_Scripting/Creating_a_GUI_for_your_script

Blender has a core written in C++ for largely same reasons Ardour needs it but has pretty amazing amounts of python power under the hood. It’s really great for workflow automation which is a big need in the 3d and video industry (Blender also has a full featured video editor inside it).

4 Likes

That’s a great perspective, @d3v8. Welcome!