Building a MinGW WoW64 Wine with a custom vkd3d build

As of version 7.4, Wine comes with a bundled version of vkd3d.

However, if you want to use a more modern version of vkd3d, apply certain patches to it, or test fixes of your own, you will need Wine to work with a custom version of vkd3d.

For this purpose, it is necessary to build both projects and link them correctly.

Compiling a Windows on Windows (WoW) version of Wine, so that it supports both 32-bits and 64-bits Windows applications – which is often desired – adds a layer of complexity to the build process.

Furthermore, you may want to use MinGW to cross-compile the Wine executables and DLLs in the Windows PE format. This is because various programs expect the on-disk DLLs to look like real PE DLLs.

This guide aims to help you through that process.

Preliminaries

Install Wine dependencies

First, install the dependencies listed in the Building Wine article.

Remember to install the :i386 version of the required packages too (if your system has multi-lib support).

The mingw and vulkan packages are of particular importance.

It is also recommended that you install ccache, as it may save us a considerable amount of time during the builds.

If you have a system-wide installation of vkd3d, it may be good to uninstall it using make uninstall. To ensure that the correct version of vkd3d is linked in this tutorial.

Directory organization

For this tutorial we will need to create several build directories and have several git repositories. For this reason, we will start by creating a common directory for them:

mkdir -p $HOME/wine-dirs

Cloning wine and vkd3d repositories

First, we clone the wine repository if we haven't already:

git clone git://source.winehq.org/git/wine.git $HOME/wine-dirs/wine-git

We also need to clone vkd3d repository if we haven't already:

git clone git://source.winehq.org/git/vkd3d.git $HOME/wine-dirs/vkd3d-git

In case you already have these repositories somewhere else, you can create symbolic links inside wine-dirs instead:

cd $HOME/wine-dirs
ln -s <relative-path-to-wine-repo> wine-git
ln -s <relative-path-to-vkd3d-repo> vkd3d-git

Now it is a good time to pick a particular branch or apply the desired patches to these repositories.

At the time of writing, the upstream Wine master branch is missing a patch that makes the d3dcompiler DLL use vkd3d's HLSL compiler for D3DCompile() and D3DCompile2(). In case you want to add this patch, execute the following commands to download this patch and apply it.

cd $HOME/wine-dirs/wine-git
wget https://source.winehq.org/patches/data/228839
git am 228839

Building

Build a provisional 64-bit Wine

In order to build 64-bit vkd3d we need some dependencies that can be built from the Wine repository. Namely, WIDL and the vulkan-1 DLLs.

For that purpose we need first a provisional 64-bit MinGW Wine installation, so we start by creating a folder:

cd $HOME/wine-dirs
mkdir wine64p

We run the Wine configure script from the installation folder

cd wine64p
../wine-git/configure --enable-win64 --disable-tests \
  CC='ccache gcc' CROSSCC='ccache x86_64-w64-mingw32-gcc'

Note that we use CROSSCC= to indicate the compiler used for cross-compilation, in this case we need the 64-bit mingw32 compiler. We prepend ccache which works as a wrapper for the compiler. The same is done for the regular C compiler, gcc, passed with the CC= variable.

Since this is just a provisional build of wine, we use --disable-tests to speed up the compilation times.

Note that to know about other parameters for the configure script we can use:

../wine-git/configure --help

Also, if something from the configure command fails, it is useful to search in config.log for the line that contains the pertinent error, and the lines before and after it.

Once the configure script has finished, we build Wine. We can pass -j<n> to use <n> threads.

make -j8

Build a 64-bit vkd3d installation

Now we need to do the 64-bit vkd3d build. For that purpose we will also need the Vulkan headers. One way of getting them is downloading the Vulkan SDK.

Alternativelly, you can get them from the official Vulkan-Headers repository. Similarly for the SPIRV-Headers if you want to build with --with-spirv-tools.

Assuming you downloaded and extracted the SDK, move it to the common directory:

mv <vulkan sdk path> $HOME/wine-dirs/vulkan-sdk

You must change the first argument of the mv command depending on where you downloaded and extracted the SDK. e.g. ~/Downloads/vulkansdk-linux-x86_64-1.3.204.1/1.3.204.1.

We have to run autogen.sh in vkd3d-git to generate vkd3d's configure script

cd $HOME/wine-dirs/vkd3d-git
bash autogen.sh

Now we create a folder for the 64-bit vkd3d build

cd $HOME/wine-dirs
mkdir vkd3d64

We run the configure script in the installation folder:

cd vkd3d64
../vkd3d-git/configure \
  --host=x86_64-w64-mingw32 \
  --prefix="$HOME/wine-dirs/vkd3d64-prefix" \
  WIDL="$HOME/wine-dirs/wine64p/tools/widl/widl" \
  VULKAN_LIBS="-L$HOME/wine-dirs/wine64p/dlls/vulkan-1/" \
  CPPFLAGS="-I$HOME/wine-dirs/vulkan-sdk/x86_64/include" \
  LDFLAGS="-static-libgcc"

• --host=x86_64-w64-mingw32 indicates that we want to cross-compile the programs and libraries using the 64-bit MinGW compiler.
• --prefix="$HOME/wine-dirs/vkd3d64-prefix" is used to set this build's installation directory. In this case, we will create a new folder inside wine-dirs. We have to keep in mind that the other programs will expect the vkd3d libraries to be in this directory at run-time.
• We set the WIDL and VULKAN_LIBS variables to point to our provisional 64-bit wine's WIDL and vulkan-1 DLL, respectively.
• We use CPPFLAGS so that the compiler knows to search in the vulkan-sdk folder for the vulkan headers (-I flag).
• We use LDFLAGS="-static-libgcc" because MinGW gcc depends on libgcc_s_sjlj-1.dll; this links a static version. Otherwise we should add the right path using the -L flag.

Then we run

make

And if the build succeeds, then we run

make install

to install 64-bit vkd3d in $HOME/wine-dirs/vkd3d64-prefix.

Build 64-bit wine

Now with the 64-bit vkd3d installation, we can compile a 64-bit Wine linked to it.

We proceed creating a new folder:

cd $HOME/wine-dirs
mkdir wine64

We run the Wine configure script again, but now specifying the parameters to link Wine to the 64-bit vkd3d prefix we just created. We will also need to include the vulkan headers:

cd wine64
../wine-git/configure --enable-win64 \
  CC='ccache gcc' CROSSCC='ccache x86_64-w64-mingw32-gcc' \
  --with-system-dllpath="$HOME/wine-dirs/vkd3d64-prefix/bin:<mingw 64-bit gcc dynamic dependencies>" \
  VKD3D_PE_CFLAGS="-I$HOME/wine-dirs/vulkan-sdk/x86_64/include -I$HOME/wine-dirs/vkd3d64-prefix/include/vkd3d" \
  VKD3D_PE_LIBS="-L$HOME/wine-dirs/vkd3d64-prefix/lib -lvkd3d -lvkd3d-shader"

• We use --with-system-dllpath so that wine knows to search our vkd3d prefix for the vkd3d DLLs, and also to search MinGW-w64's gcc Win64 dynamic dependencies.

  Notice that both directories are separated with a colon.

  You need to replace <mingw 64-bit gcc dynamic dependencies> with the right directory. In Ubuntu, the directory is /usr/lib/gcc/x86_64-w64-mingw32/9.3-win32 and the dependencies are provided by the gcc-mingw-w64-x86-64 package.

• We set VKD3D_PE_CFLAGS and VKD3D_PE_LIBS to indicate that we will use a custom version of vkd3d. We use VKD3D_PE_CFLAGS to include the vulkan headers and the vkd3d headers and VKD3D_PE_LIBS to load vkd3d's dynamic libraries.

If you get the following message around the last lines:

configure: libvkd3d 64-bit MinGW development files not found; using bundled version.

Wine was not correctly linked to your custom version of vkd3d. Check the config.log.

Then we build Wine again:

make -j8

Build a provisional 32-bit Wine

Now that we have a working 64-bit Wine we need to build a 32-bit Wine and link it to the 64-bit Wine in order to support 64-bit applications. For that purpose, we need to repeat the previous steps but now with a 32-bit Wine build. Starting by creating a provisional 32-bit Wine installation in order to compile a 32-bit vkd3d.

cd $HOME/wine-dirs
mkdir wine32p

cd wine32p
../wine-git/configure --disable-tests \
  CC='ccache gcc' CROSSCC='ccache i686-w64-mingw32-gcc'

make -j8

Build a 32-bit vkd3d installation

cd $HOME/wine-dirs
mkdir vkd3d32

cd vkd3d32
../vkd3d-git/configure \
  --host=i686-w64-mingw32 \
  --prefix="$HOME/wine-dirs/vkd3d32-prefix" \
  WIDL="$HOME/wine-dirs/wine32p/tools/widl/widl" \
  VULKAN_LIBS="-L$HOME/wine-dirs/wine32p/dlls/vulkan-1/" \
  CPPFLAGS="-I$HOME/wine-dirs/vulkan-sdk/x86_64/include" \
  LDFLAGS="-static-libgcc"

• We now use --host=i686-w64-mingw32 to indicate that we will use the 32-bit MinGW compiler.
• We are changing the prefix so that this 32-bit vkd3d is installed in the vkd3d32-prefix folder instead of vkd3d64-prefix.
• We also will link to the WIDL and vulkan-1 DLLs inside our 32-bit provisional wine.
• We can still use the vulkan headers in our 64-bit vulkan-sdk.

We then run:

make

make install

Build 32-bit Wine

We can now build our 32-bit Wine, linking it to our 32-bit vkd3d.

cd $HOME/wine-dirs
mkdir wine32

cd wine32
../wine-git/configure --with-wine64=../wine64/ \
  CC='ccache gcc' CROSSCC='ccache i686-w64-mingw32-gcc' \
  --with-system-dllpath="$HOME/wine-dirs/vkd3d32-prefix/bin:<mingw 32-bit dynamic dependencies>" \
  VKD3D_PE_CFLAGS="-I$HOME/wine-dirs/vulkan-sdk/x86_64/include -I$HOME/wine-dirs/vkd3d32-prefix/include/vkd3d" \
  VKD3D_PE_LIBS="-L$HOME/wine-dirs/vkd3d32-prefix/lib -lvkd3d -lvkd3d-shader"

• We now point --with-system-dllpath and VKD3D_PE_LIBS to the directories inside vkd3d32-prefix.

  You also have to replace <mingw 32-bit dynamic dependencies> with MinGW-w64's gcc Win32 dynamic dependencies' directory now. In Ubuntu this directory is /usr/lib/gcc/i686-w64-mingw32/9.3-win32 and the dependencies are provided by the gcc-mingw-w64-i686 package.

Finally, build Wine:

make -j8

Updating your build

If you want to update either wine or vkd3d's source code, we can execute the following commands in order. ccache should help reducing the compilations times.

# Recompile 64-bit vkd3d
cd $HOME/wine-dirs/vkd3d64;
make
make install

# Recompile 64-bit wine
cd $HOME/wine-dirs/wine64;
make -j8

# Recompile 32-bit vkd3d
cd $HOME/wine-dirs/vkd3d32;
make
make install

# Recompile 32-bit wine
cd $HOME/wine-dirs/wine32;
make -j8

Sometimes, for instance if you change the --with-system-dllpath argument, you would also need to execute make clean before.