Build Ardour with windows-vst support

jensr · March 18, 2020, 4:11pm

I’ve tested Ardour on Linux with various Windows VST’s with Carla and LinVst.

Now I want to test Ardour’s built-in support for Windows VST. First, I want to know if anyone has been able to compile Ardour with this configuration flag:

./waf configure --windows-vst [...]

I’ve tried in Fedora, Ubuntu and CentOS with different versions of Wine.

Yes, I know it is unsupported, not recommended, dangerous etc. But right now I can spend time on this.

x42 · March 18, 2020, 5:03pm

How does it fail?

Maybe simply winegcc and wineg++ are missing in $PATH? (on debian they’re in /usr/lib/wine/)

Edit: maybe try

ln -s /usr/bin/winegcc-stable /usr/bin/winegcc
ln -s /usr/bin/winegcc-stable /usr/bin/wineg++

jensr · March 18, 2020, 5:33pm

Thanks! Here is my “recipe” for Fedora:

docker run -it fedora:31

dnf -y install 'dnf-command(builddep)' git wine-devel
dnf -y builddep ardour5

git clone https://github.com/Ardour/ardour.git
cd ardour

python3 waf configure --windows-vst
python3 waf

Here is the error message:

[...]
winegcc: gcc failed

In file included from /usr/include/wine/windows/windows.h:64,
                 from ../libs/fst/vstwin.c:24:
/usr/include/wine/windows/winsock.h:535:15: error: unknown type name ‘u_short’
  535 | static inline u_short __wine_ushort_swap(u_short s)
      |               ^~~~~~~
/usr/include/wine/windows/winsock.h:535:42: error: unknown type name ‘u_short’; did you mean ‘short’?
  535 | static inline u_short __wine_ushort_swap(u_short s)
      |                                          ^~~~~~~
      |                                          short
/usr/include/wine/windows/winsock.h: In function ‘__wine_ulong_swap’:
/usr/include/wine/windows/winsock.h:541:40: error: ‘u_short’ undeclared (first use in this function); did you mean ‘WS_u_short’?
  541 |     return ((ULONG)__wine_ushort_swap((u_short)l) << 16) | __wine_ushort_swap((u_short)(l >> 16));
      |                                        ^~~~~~~
      |                                        WS_u_short
/usr/include/wine/windows/winsock.h:541:40: note: each undeclared identifier is reported only once for each function it appears in
/usr/include/wine/windows/winsock.h:541:48: error: expected ‘)’ before ‘l’
  541 |     return ((ULONG)__wine_ushort_swap((u_short)l) << 16) | __wine_ushort_swap((u_short)(l >> 16));
      |                                                ^
      |                                                )
../libs/fst/vstwin.c: At top level:
../libs/fst/vstwin.c:48:10: fatal error: glib.h: No such file or directory
   48 | #include <glib.h>
      |          ^~~~~~~~
compilation terminated.
winegcc: gcc failed

Waf: Leaving directory `/ardour/build'
Build failed
 -> task in 'ardour-vst-scanner.exe.so' failed with exit status 2 (run with -v to display more information)
 -> task in 'ardour-6.0.pre0.3672-vst.exe.so' failed with exit status 2 (run with -v to display more information)

I have tried using glib.h from “mingw64-glib2” and “glib-devel” (also copied to different places) but it just gives me more errors.

jensr · March 18, 2020, 7:08pm

… and this is for Ubuntu 18.04:

docker run -it ubuntu:18.04

sed -i 's|^# deb-src|deb-src|g' /etc/apt/sources.list
apt-get update

apt-get -y build-dep ardour
apt-get -y install git libwine-dev libjack-dev

git clone https://github.com/Ardour/ardour.git
cd ardour
./waf configure --windows-vst
./waf

Error:

../libs/fst/vstwin.c:48:10: fatal error: glib.h: No such file or directory
 #include <glib.h>
          ^~~~~~~~
compilation terminated.
winegcc: gcc failed

Waf: Leaving directory `/home/user/ardour/build'
Build failed
 -> task in 'ardour-6.0.pre0.3674-vst.exe.so' failed with exit status 2 (run with -v to display more information)

I have /usr/include/glib-2.0/glib.h from libglib2.0-dev.

Next try:

./waf configure --windows-vst --also-include=/usr/include/glib-2.0
./waf

Error:

/usr/include/glib-2.0/glib/gtypes.h:32:10: fatal error: glibconfig.h: No such file or directory
 #include <glibconfig.h>
          ^~~~~~~~~~~~~~
compilation terminated.
winegcc: gcc failed

Waf: Leaving directory `/home/user/ardour/build'
Build failed
 -> task in 'ardour-6.0.pre0.3674-vst.exe.so' failed with exit status 2 (run with -v to display more information)

I can not “split multiples with commas” (from ./waf --help) to include the path to glibconfig.h. It just takes me back to the first error (glib.h):

 ./waf configure --also-include=/usr/include/glib-2.0,/usr/lib/x86_64-linux-gnu/glib-2.0/include --windows-vst

x42 · March 18, 2020, 7:32pm

Try Ardour 5.12 instead, that’s know to work at least.
Also keep in mind that this was only ever used/tested with 32bit bit builds, loading 32 bit windows VST.
A 64bit version should compile, but as far as I can tell, nobody ever used it. Mainly because LinVST took over.

I’ve pushed a small change to Ardour-git just now, that should fix the glib issue. At least it compiles and links here. I don’t have any Windows VST to test.

jensr · March 18, 2020, 8:18pm

Amazing! Now I can compile Ardour 6 in Ubuntu 18.04. Fedora doesn’t work, but it may have to do with Wine (I’ll check that later).

jensr · March 19, 2020, 8:00pm

I can compile with --windows-vst in Ubuntu 18.04, 19.10 and 20.04. I have to use --wine-include in Ubuntu 19.10 and 20.04:

waf configure --wine-include=/usr/include/wine/wine/windows [...]

In Fedora, I can’t compile. I’ve tested in Fedora 27, 30, 31 and 32 (beta) (to get different versions of Wine). Here is the output from Fedora 31:

docker run -it fedora:31

dnf -y install 'dnf-command(builddep)' git wine-devel
dnf -y builddep ardour5

git clone https://github.com/Ardour/ardour.git
cd ardour

python3 waf configure --windows-vst
python3 waf

In file included from /usr/include/wine/windows/windows.h:64,
                 from ../libs/fst/vstwin.c:24:
/usr/include/wine/windows/winsock.h:535:15: error: unknown type name ‘u_short’
  535 | static inline u_short __wine_ushort_swap(u_short s)
      |               ^~~~~~~
/usr/include/wine/windows/winsock.h:535:42: error: unknown type name ‘u_short’; did you mean ‘short’?
  535 | static inline u_short __wine_ushort_swap(u_short s)
      |                                          ^~~~~~~
      |                                          short
/usr/include/wine/windows/winsock.h: In function ‘__wine_ulong_swap’:
/usr/include/wine/windows/winsock.h:541:40: error: ‘u_short’ undeclared (first use in this function); did you mean ‘WS_u_short’?
  541 |     return ((ULONG)__wine_ushort_swap((u_short)l) << 16) | __wine_ushort_swap((u_short)(l >> 16));
      |                                        ^~~~~~~
      |                                        WS_u_short
/usr/include/wine/windows/winsock.h:541:40: note: each undeclared identifier is reported only once for each function it appears in
/usr/include/wine/windows/winsock.h:541:48: error: expected ‘)’ before ‘l’
  541 |     return ((ULONG)__wine_ushort_swap((u_short)l) << 16) | __wine_ushort_swap((u_short)(l >> 16));
      |                                                ^
      |                                                )
In file included from ../libs/fst/fst.h:29,
                 from ../libs/fst/vstwin.c:49:
../libs/ardour/ardour/vst_types.h: At top level:
../libs/ardour/ardour/vst_types.h:34:1: warning: ‘visibility’ attribute ignored on types [-Wattributes]
   34 | };
      | ^
../libs/ardour/ardour/vst_types.h:57:1: warning: ‘visibility’ attribute ignored on types [-Wattributes]
   57 | };
      | ^
../libs/ardour/ardour/vst_types.h:75:1: warning: ‘visibility’ attribute ignored on types [-Wattributes]
   75 | };
      | ^
../libs/ardour/ardour/vst_types.h:134:1: warning: ‘visibility’ attribute ignored on types [-Wattributes]
  134 | };
      | ^
../libs/fst/vstwin.c: In function ‘fst_run_editor’:
../libs/fst/vstwin.c:432:15: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
  432 |    fst->xid = (int) GetPropA (fst->windows_window, "__wine_x11_whole_window");
      |               ^
At top level:
../libs/fst/vstwin.c:61:25: warning: ‘magic’ defined but not used [-Wunused-const-variable=]
   61 | static const char       magic[]          =  "FST Plugin State v002";
      |                         ^~~~~
winegcc: gcc failed

Waf: Leaving directory `/ardour/build'
Build failed
 -> task in 'ardour-vst-scanner.exe.so' failed with exit status 2 (run with -v to display more information)
 -> task in 'ardour-6.0.pre0.3676-vst.exe.so' failed with exit status 2 (run with -v to display more information)

Can anyone guide me in the right direction? Thanks!

x42 · March 19, 2020, 9:09pm

Does it run, and load VSTs? cd vst; ./ardevst

jensr · March 20, 2020, 12:17pm

I have all my tools on the other side (redhat) It is possible to start and use, but I have not tested any VST. I will come back when I have had the opportunity to test.

peder · March 20, 2020, 12:35pm

The u_short error seems to be a bug in the Fedora winsock.h file; unless there’s some other thing going on higher up in the compile chain.

Try adding
#ifndef USE_WS_PREFIX
below
/* we have to define hton/ntoh as macros to avoid conflicts with Unix headers */
in that file and also
#endif /* USE_WS_PREFIX */
below
#define ntohs(s) __wine_ushort_swap(s)
#endif /* WORDS_BIGENDIAN */

Ardour defines USE_WS_PREFIX, as seen in Robin’s fix, and Ubuntu 18.04’s winsock.h has that ifndef to make sure it doesn’t use the u_short type.

mon0 · March 20, 2020, 7:32pm

It is possible to compile in CentOS 8 with Wine (4-staging) from EPEL. If I use Wine 5.4-staging (manually installed) I get ‘u_short errors’.

Here is my Ardour log from Ubuntu 18.04 (ardevst compiled with the instructions in this thread):

2020-03-20T14:15:13 [INFO]: --- Windows VST plugins Scan: /home/user/.wine/drive_c/Program Files/Native Instruments/VSTPlugins 64 bit:/home/user/.wine/drive_c/Program Files/Vstplugins:/home/user/.wine/drive_c/Program Files/Steinberg/VSTPlugins:/usr/local/lib/vst:/usr/lib/vst
2020-03-20T14:15:13 [INFO]:  *  /home/user/.wine/drive_c/Program Files/Native Instruments/VSTPlugins 64 bit/Kontakt.dll - x64 (64-bit)
2020-03-20T14:15:14 [INFO]:  -> Cannot get Windows VST information, plugin ignored.
2020-03-20T14:15:14 [INFO]:  *  /home/user/.wine/drive_c/Program Files/Vstplugins/SINE Player.dll - x64 (64-bit)
2020-03-20T14:15:14 [INFO]:  -> Cannot get Windows VST information, plugin ignored.
2020-03-20T14:15:14 [INFO]:  *  /home/user/.wine/drive_c/Program Files/Steinberg/VSTPlugins/Pianoteq 6 STAGE (64-bit).dll - x64 (64-bit)
2020-03-20T14:15:14 [INFO]:  -> Cannot get Windows VST information, plugin ignored.
2020-03-20T14:15:14 [INFO]: --- Windows VST plugins Scan Done

SINE Player.dll works well with LinVst and Ardour 5.12. But I’ve only tested with Wine 5.3 and 5.4 (staging).

update:
Now I’ve tried the same way with Ardour 6 in CentOS 8 with Wine 5.4-staging. I can compile with @peder’s fix:

sed -i '513i#ifndef USE_WS_PREFIX' /usr/include/wine/windows/winsock.h
sed -i '550i#endif /* USE_WS_PREFIX */' /usr/include/wine/windows/winsock.h

When I do a Windows VST scan, I get the same output in the Ardour log:

Cannot get Windows VST information, plugin ignored.

Everything else I’ve been able to test works just as well from ./ardevst (compared to my Ardour 5.12) (midi-keyboard, Pianoteq LV2, calf-plugins, etc.).

peder · March 21, 2020, 12:43pm

As Robin says earlier in this thread : “keep in mind that this was only ever used/tested with 32bit bit builds, loading 32 bit windows VST”.
Your using 64-bit dlls could be the problem.

mon0 · March 21, 2020, 9:29pm

After some debugging:

github.com

Ardour/ardour/blob/386264bd235a042d5d46550b04f6bdbdc3aac54b/libs/ardour/vst_info_file.cc#L1002




		char **argp= (char**) calloc (3,sizeof (char*));
		argp[0] = strdup (scanner_bin_path.c_str ());
		argp[1] = strdup (dllpath);
		argp[2] = 0;


		set_error_log (dllpath);
		ARDOUR::SystemExec scanner (scanner_bin_path, argp);
		PBD::ScopedConnectionList cons;
		scanner.ReadStdout.connect_same_thread (cons, boost::bind (&parse_scanner_output, _1 ,_2));
		if (scanner.start (ARDOUR::SystemExec::MergeWithStdin)) {
			PBD::error << string_compose (_("Cannot launch VST scanner app '%1': %2"), scanner_bin_path, strerror (errno)) << endmsg;
			close_error_log ();
			return infos;
		} else {
			int timeout = PLUGIN_SCAN_TIMEOUT;
			bool no_timeout = (timeout <= 0);
			ARDOUR::PluginScanTimeout (timeout);
			while (scanner.is_running () && (no_timeout || timeout > 0)) {
				if (!no_timeout && !ARDOUR::PluginManager::instance ().no_timeout ()) {
					if (timeout%5 == 0) {

parse_scanner_output () msg:

01e3:err:module:__wine_process_init failed to load L"Z:\home\user\git\ardour\build\libs\fst\ardour-vst-scanner.exe.so", error c000012f

The same error is displayed if I run the scanner manually.

scanner_bin_path: /home/user/git/ardour/build/libs/fst/ardour-vst-scanner

x42 · March 21, 2020, 9:34pm

Which is why I suggested to try 5.12 (and ideally 32bit with 32bit VSTs).
With Ardour6/git there will be are at least 3 or 4 follow up errors, even if you’d fix the scanner wrapper.

mon0 · March 21, 2020, 10:15pm

This is my fixed wrapper:

#!/bin/sh

TOP=`dirname "$0"`/../../..
. $TOP/build/gtk2_ardour/ardev_common_waf.sh
export LD_LIBRARY_PATH=$libs/../gtk2_ardour:$LD_LIBRARY_PATH

exec wine "`dirname "$0"`/ardour-vst-scanner.exe.so" "$@"

But I’ll listen to you and give up.

armislv · June 24, 2020, 7:57pm

Yes, I was able to build Ardour 6 with built-in Windows VST support on Fedora 31. Seems it fails to build because of broken winsock code in the wine, so I added #define WINE_NOWINSOCK before #include <windows.h> in the libs/fst/vstwin.c and then it builds. But… does not start
It dies with:
Unhandled exception: assertion failed, invalid program stack in 64-bit code (0x00007fd3ba511625).