Just one more thing… when I was trying your vcpkg, libffi was in fact building and installing but something was breaking immediately afterwards. I assumed vcpk must run some cleanup process at the end of a build and maybe that’s where it was all going wrong.
@EZ4Steohen - I managed to find some time yesterday to run your command with the official version of vcpkg and everything builds fine. The only drawback of course is that Classic mode (IIUC) has no knowledge of earlier package versions - i.e. it only ever builds the most recent version. But having said that, Manifest mode isn’t a whole lot better in my experience.
I haven’t tried your modified repo but I what I’m seeing here is identical to what I’d already seen with my own attempts (so if I’m honest, I suspect your repo will probably still bail out at libffi). Let me know if you need me to try and I’ll see if I can fit it in either today or tomorrow.
[Edit…] I just gave your repo a very quick try but your command line isn’t accepted (it tells me that command line arguments aren’t allowed). So it looks like your repo does in fact build in Manifest mode.
[Edit 2…] Another idea just occurred to me… as an experiment, might it be worthwhile for you to create a version of your repo with libffi already pre-built?
Regarding manifest not being better: In this particular case, I agree. Using vcpkg manifest was initially part of the design to install the libs because I was making use of vcpkg’s ports design to install old versions of 4 dependencies. When I removed and transferred that function to the python script, the manifest file was essentially a slightly complex version of the vcpkg classic mode, with the known difference of where the libraries are built.
Assuming this implies no errors happened during the vcpkg classic command’s installation: I’d say Step 1 is complete. What remains should be libsigcpp, glibmm, cairomm, and pangomm, right?
Yes, I need to delete it and replace it with 2/3 commands. Now if I’m not mistaken, the Python script needs edits relating to the folder that was previously vcpkg_installed now being the folder “installed” in wherever you installed the vcpkg repo.
The idea feels risky… I’m not particularly thrilled at the idea of somehow sharing binaries (among other things) like that. Maybe it’s my inexperience in such matters, but I’ll not do that.
And I almost forgot, perl is also needed.
TL;DR: With the other deps being successfully installed, what remains is the python script to install the remaining 4. And perl is also needed for one step in the build.
I made some edits to the Python script to fix some errors relating to vcpkg_installed vs vcpkg/installed. Put it at the same level as where the vcpkg folder (that was cloned from github) is located, then run it. python InstallOthers.py
Do tell if any errors arise, because on my end the edits made it work without any issue.
Yes they’re the main ones. I’ve a small number of extra libs that I build here, although they might not be needed any more:- libart_lgpl / raptor / freeimage and aubio
BTW, one strange thing I noticed was how much of libboost seems to get built as DLLs with vcpkg. Locally I only need to build boost-regex as a DLL. Everything else is useable from the boost header files. That’s not your fault of course - it’s just an oddity that I noticed.
[Edit…] Oh, BTW:-
I was only suggesting that as an experiment. Given that we can’t figure out what the problem is, I thought it’d be useful to see if it allows me to get further with the build.
@John_E Regarding installing most of the dependencies using vcpkg classic mode: libffi is included, right? I recently realized I didn’t try to make use of them using vcpkg classic mode for the last 4 dependencies (For some reason I didn’t check whether it’s possible to use ports with vcpkg classic), and sometime within the next two weeks I want to re-attempt vcpkg’s port overlay mechanic but for vcpkg classic.
(Part of the reason I’m revisiting the installation of dependencies, or rather the 4 that give some problems, is that for some reason, when building and installing ardour, ardour was suddenly looking for glibmm-2.4 and giomm-2.4 instead of glibmm-2.4.dll and giomm-2.4.dll. (and I suspected it wasn’t ardour’s fault.) Made me rethink the reliance of the python script and check if port overlays worked with vcpkg classic all along)
Could there be a way to test the overlays from my repo from an older commit? Maybe, but I’ll not look into that just yet, as I want to get my laptop screen fixed before looking into that.
@EZ4Stephen I’m very keen to avoid giving you the wrong impression so it’s probably time for a reality check here… when I first got involved in this, my understanding was that your goal was to simplify the build process in the hope it might help to attract future programmers. But whatever else your solution achieves it certainly doesn’t simplify anything - not least because vcpkg is too inconsistent when dealing with earlier library versions. Please be assured that I’m not in any way trying to belittle the determination you’ve shown, which is very impressive but I can’t imagine any other programmer having the patience to figure it all out if they encounter similar hurdles.
Truth is I’ve never believed vcpkg was the right tool for a task like this and it’s not a path I want to continue along.
What makes you think so? From what I gather it is the right tool for open source projects on Windows with direct MSVC integration. Is there another library/package tool to consider?
It’s rather long-winded but here goes… on Windows, MSVC can use different compilers. The most common one is MSVC’s internal compiler and it can also use Clang. But apart from the compiler it also needs a build system (i.e. something to tell the compiler what to do). Locally I build Ardour and all its support libs using VS project files (.vcxproj). For open source code, VS project files are almost non-existent these days but if someone’s skilled enough to produce them, they’re compatible with any version of a library. However, vcpkg doesn’t use VS project files. vcpkg uses two other build systems:- one is CMake and the other is called Meson.
Unfortunately, there are problems if Ardour’s support stack needs some earlier library versions. Firstly, not all the earlier versions are available from vcpkg. So if we needed version 2.55 of some given library, vcpkg might only offer version 2.42 or version 2.68. And there’s a further complication… Meson seems to have no concept of backwards compatibility so if that library’s build system is Meson and you need to build some earlier version, Meson almost always complains that the earlier version is no longer buildable. The bottom line is that vcpkg is a truly excellent build system - but it’s only guaranteed to work properly if you build the latest versions of everything.
A further issue is that vcpkg isn’t set in stone. It’s constantly evolving. So if Stephen builds with vcpkg now and then some other programmer decides to build with it sometime next year, their support stack will be even further misaligned. The only guaranteed ways to keep everything in step are:-
a) Ardour expands 'libs/tk' to include the other requirements (glib / cairo / freetype etc)
b) I update my libs to the latest versions needed (by Ardour) and build via .vcxproj files
c) You (Robin) supply a pre-built support stack, built using Clang
If Stephen could produce vcpkg libraries which aren’t necessarily the correct ones but they’re “close enough” AND he then releases them as a pre-built support stack, that could also work. But like I said earlier, if any of his libs need to get built using Meson, there’s no guarantee they’ll still be buildable in the future.
So in a nutshell, if the goal is to produce a compatible support stack that’s also future-proof, there are various ways to achieve it but vcpkg isn’t one of them.
I wouldn’t describe my efforts as “simplify the build process”, but rather make the existing main build system work fully with MSVC, though the objective of attracting (Windows) programmers is correct. Like, why wouldn’t I want someone to come along and make what I use objectively better?
No worries.
Ah well… I still appreciate your patience with me and vcpkg to have tried until the latest request nonetheless.
I haven’t looked deeply into every competitor/substitute to vcpkg besides Conan. (Ignoring libsigcpp/glibmm/pangomm/cairomm since older versions are required, and this may make things difficult), Conan lacks (at least) 9 dependencies.
Maybe Conda and Xmake/Xrepo are alternatives, but I only checked for like 1-4 packages, and neither had librubberband. I can certainly put in effort to look at Conda, seeing as how they seem to have the right versions of libsigcpp & Co. available.
Also, theoretically one could make some script for getting all dependencies, but such a script might be less auditable for anything malicious compared to using vcpkg/conan. (Ignoring updating any dependencies regardless of whether updating any installed dependency will be needed for Ardour)
I have ideas of adding support to build Ardour using VS project/solution files using waf, though that’ll not be added just yet. (Gotta figure out the details of how the files would need to be written, and I haven’t looked into the nuances such as the options passed through waf. And I put time towards things other than Ardour too, so when I choose to look into that, I’m not sure…)
Agreed. But does there exist a library/package manager that supports all earlier versions with no gaps? I doubt it. Seems like an inherent drawback of all package managers, if you ask me.
Probably the point hardest to refute, from personal experience.
Acknowledged.
Given I feel like I could look into Conda (and maybe Xrepo): unless I’m absolutely certain that I have a vcpkg solution that just works fully, you won’t be bothered by requests from me about vcpkg. But, if I want your involvement to try downloading from a package manager that isn’t vcpkg, would you be willing to try it, or do you want to avoid being the one who tests ways to get the dependencies altogether, until there there is a complete enough solution? Like, if you feel your time and efforts can be put elsewhere, I’ll take your word and not insist upon your participation.
(Though eventually someone has to repeat the steps I take to get Ardour’s deps, to prove the process is doable on Windows, whatever those steps will be… When that time comes, I’ll probably make a new post than post here.)
Also, a separate post will be made regardless of whether the dependencies issue has a clear solution, when Ardour itself becomes buildable with MSVC via waf without any edits required to Ardour’s source code. It’s not far…
We won’t do that. The main goal here is to provide a way that allows potential contributors to build it from scratch, using the compiler they want. Just like on Linux and macOS.
The goal is the opposite. We do not want to provide a binary support stack.
I can only speak for Windows. Over the years I’ve helped maybe 7 or 8 MSVC programmers who wanted to get involved in building Ardour. Not one person ever got anywhere near building Ardour itself. Every single one threw in the towel because it was just too difficult to build (and in most cases, even find) the large number of support libs needed.
I’d like to see you and Stephen find a way to improve that process but this isn’t the route that I’d favour so I’d prefer to be left out of it.
1 Like
It’d be similar on other OS were it not for package managers, as provided by GNU/Linux distros, and in case of macOS: homebrew and MacPorts, etc. On those OS, it is pretty much a one-liner to get all Ardour build dependencies.
While there is Cygwin and MSYS2/pacman for MinGW, Windows has lacked similar software for MSVC until recently.
Given that those tools have to catch up 30 years, I don’t expect a one-liner, but I expect that a relatively simple solution using Conan2 or vcpkg or similar will be possible.
All that matters is that two stood against many. [Conan, 1982]
That’s a good point. Robin - what’s the situation on MacOS and Linux? Do their package managers provide the correct versions needed for Ardour or is it a case of building packages that are “close enough”?
That’s another good point. All those years ago (back when I first proved that Ardour was even buildable on Windows) my first build was achieved using Cygwin. But Cygwin’s support was appalling and it was Ben Loftis who encouraged me to try MSVC. MSVC was orders of magnitude more difficult but (with no disrespect to MinGW) MSVC has been pretty respectable for a very long time (despite its occasional quirks!)
@EZ4Stephen - another thought… for Conda / Conan / Xrepo, did you ever find out what build system they use? If my experience is anything to go by, Meson is the gigantic weak link in vcpkg. So it might well be worthwhile for you to investigate some others if they aren’t using Meson. Also…
That doesn’t seem like a lot to add. Were they unwilling to add them for some reason?
I checked Conda today, it didn’t have some libraries. I’ll likely pivot from Ardour temporarily to submit build recipes to them.
Will check Xrepo/Xmake tomorrow. Fun fact, Xmake supports getting libraries from vcpkg, Conan, Conda, and many other package managers, so at least for short term, it’ll likely be the replacement for vcpkg.
(Also, Spack exists as a package manager, but I’ll ignore that for now.)
1 Like
just close enough. You do not need the exact same version as we use, just API compatible ones.
On my main Linux dev system I also use whatever debian provides (which is not the exact same versions that we use for binary releases).
Pretty much all GNU/Linux distros build Ardour with whatever library version they have. This can lead some subtle bugs, usually on rolling release distros that ship bleeding edge libs, but it doesn’t prevent one from compiling and running Ardour.
That’s what I thought. My own libraries aren’t identical to yours but they’re close enough and usually highlight the same bugs.
@EZ4Stephen This is probably a dumb idea but I’ll suggest it in the event all else fails…
[Edit…] Ignore me. I realised later that it was a dumb idea!!
1 Like
Regarding Conda: Looks like Conda’s build system is primarily for building Release builds. This limitation can probably be overcome by making a script, but that might be too much effort compared to just using vcpkg + ports, at the least. (Yet to check how suitable Xmake/Xrepo is, to compare.)
Will give an update on Xmake/Xrepo once I check and try using that.
Xrepo is missing 13/31 dependencies (assuming those not in the 31 that are needed by Ardour are transitively installed), after having looked at a list I made from cross-referencing my vcpkg.json and the other python script.
Ideally, I wouldn’t need to use anything other than Xrepo, and the things marked # by me would lose the # prefix:
add_requires("glib")
add_requires("libsndfile")
add_requires("curl")
add_requires("libarchive")
#add_requires("liblo")
add_requires("taglib")
#add_requires("vamp-sdk")
add_requires("libusb")
#add_requires("rubberband")
add_requires("jack2")
add_requires("pthreads4w")
add_requires("fftw")
#add_requires("aubio")
add_requires("libpng")
add_requires("pango")
#add_requires("lv2")
add_requires("libxml2")
add_requires("cppunit")
add_requires("libwebsockets")
add_requires("portaudio")
add_requires("libsamplerate")
#add_requires("serd")
#add_requires("sord")
#add_requires("sratom")
#add_requires("lilv")
add_requires("wingetopt")
add_requires("boost")
#add_requires("libsigcplusplus")
#add_requires("glibmm")
#add_requires("cairomm")
#add_requires("pangomm")
First thoughts are, I’ll get libsigcpp/glibmm/cairomm/pangomm from Conda, and the other unavailable ones (liblo, vamp-sdk, rubberband, aubio, lv2, serd, sord, sratom, lilv) from vcpkg.
That’s the same “dumb idea” I came up with earlier!! I’m not saying it won’t work but you’d need to be very careful for this reason…
Let’s say that glib needs pthreads just as an example. If you’re building glib from Xrepo, then Xrepo will build against its own version of pthreads. And if glibmm gets built from Conda, then Conda will build against its version of pthreads. So unless they both happen to build the same version of pthreads, you’ll likely encounter runtime issues that’ll be very difficult to resolve.
And the same would be true for any other modules that share external dependencies. Like I said earlier, it isn’t impossible to make it work but you’d need to think it all out very carefully.