Wine TestBot VMs
- 1 WineTestBot VMs
- 2 Virtualization
- 3 VM hardware and drivers
- 4 Windows configuration
- 5 Missing Windows dlls
- 5.1 atl80
- 5.2 atl90
- 5.3 atl100, msvcp100, msvcr100
- 5.4 atl110, msvcp110, msvcr110
- 5.5 msvcp120, msvcr120
- 5.6 msvcp140, msvcr140, ucrtbase
- 5.7 d3d8
- 5.8 d3dcompiler_43, d3dx9_36, xinput1_3
- 5.9 d3dxof
- 5.10 dmband, dmime, dplayx
- 5.11 localspl, localui, spoolss
- 5.12 msxml4
- 5.13 msxml6
- 5.14 qmgr
- 5.15 services.exe
- 5.16 xaudio2_7, xaudio2_8
- 6 Unix configuration
This page describes the new WineTestBot VMs and how they are configured.
Here is a list of libvirt bugs relevant to the WineTestBot:
- Libvirt bug 910711 - snapshot-revert causes spurious hardware mismatch errors
- This may actually be a QEMU bug as it's the switch from QEMU 1.1.2 to 1.3.90 which seems to trigger this issue.
- virt-manager bug 912810 - VM display is garbled if in 16-bpp
QEMU / KVM
Here is a list of QEMU / KVM bugs relevant to the WineTestBot:
- QEMU bug 1119281 - The virtio network device breaks !UuidCreateSequential().
- QEMU bug 930962 - Bad checksum errors caused by a bad interaction between the virtio network device and the dhcp server. Not necessarily a big issue but pollutes the logs.
- QEMU bug 1119861 - Poor console performance in Windows 7.
- We worked around this one by reducing the console buffer size in kernel32:console.
- QEMU bug 1123975 - QEMU 1.3.90 cannot restore a 1.1.2 live snapshot.
- QEMU bug 1119686 - Incorrect handling of icebp.
Spice QXL driver
Here is a list of the Spice QXL driver bugs relevant to the WineTestBot:
- QXL bug 61123 - spice-guest-tools-0.3.exe does not install the QXL driver on Vista
- QXL bug 61124 - Transparency / alpha blending bug on Vista
- QXL bug 61227 - The QXL driver reports X1R5G5B5 on XP instead of R5G6B5
- See also the wine-devel post.
VM hardware and drivers
|2000||ok||broken (1)||no driver||broken (2)|
|XP||52 test failures||60 test failures||no driver||broken (3)|
|2003||ok||no driver||no driver||no driver|
|Vista||ok||3 test failures||no driver||no driver|
|Vista 64-bit||no driver||no driver||broken (4)||no driver|
|2008 64-bit||no driver||no driver||broken (5)||no driver|
|7||no driver||no driver||ok||no driver|
|7 64-bit||no driver||no driver||ok||no driver|
- With the es1370 driver dsound:capture causes Windows 2000 to freeze.
- Windows 2000 does not detect the Sound Blaster 16 card but its driver can be installed manually. However the driver then fails to load with code 10.
- Windows XP does not detect the PCI Sound Blaster 16 card but its driver can be installed manually. However that then breaks the network cards (rtl8139, e1000 and virtio).
- The High Definition Audio Device driver (6.0.6000.16386) fails to load with code 10.
- The High Definition Audio Device driver (6.0.6001.18000) fails to load with code 10.
- pcspk does not count as a sound card.
|2000||317 test failures||vga fallback (1)||vga fallback (1)||vga fallback (1)|
|XP||236 test failures||117 test failures (2)||empty driver name||ok|
|2003||ok (5, 7)||ok||ok||ok (5)|
|Vista||vga fallback; 234 test failures||ok (2)||ok||ok (3)|
|Vista 64-bit||vga fallback; 233 test failures||1 test failure||ok||ok (4)|
|2008 64-bit||vga fallback; 233 test failures||broken (6)||ok||ok (4)|
|7||vga fallback; 234 test failures||ok (2)||ok||ok (4)|
|7 64-bit||vga fallback; 234 test failures||ok (2)||ok||ok (4)|
- The Windows 2000 fallback driver only support the standard VGA mode: 640x480x4.
- Using the 'Red Hat QXL GPU' 22.214.171.12416 driver.
- Using the 'VMware SVGA II' 126.96.36.199 XPDM driver from VMware Tools 9.0.1-913578.
- Using the 'VMware SVGA II' 188.8.131.52 XPDM driver from VMware Tools 8.3.17-870839.
- Using the default VGA driver.
- The 'Red Hat QXL GPU' 184.108.40.20616 driver fails to load with code 39.
- Operates in 24bpp only.
- None of the Windows versions has a xen driver.
|2000||no driver||ok (1)||ok||crash (2)||no driver|
|XP||no driver||ok||ok||ok||1 test failure|
|Vista 64-bit||ok||no driver||no driver||ok||1 test failure|
|2008 64-bit||ok||no driver||no driver||ok||1 test failure|
|7||ok||no driver||no driver||ok||1 test failure|
- With the ne2k_pci card network transfers cause high CPU usage and are slow (~10Mbps).
- The rtl8139 card causes Windows 2000 to crash on startup.
Ideally all VMs would be configured with two processors by default. However this currently seems to trigger quite a few race conditions in our tests (e.g. in ws2_32:sock) so they have currently all been configured with only one processor, except for the w864 VM. This issue will be revisited when the WineTestBot is in service.
Windows 10 needs the High Precision Event Timer (HPET) otherwise it causes a high CPU usage on the host. Use virsh edit and in the <clock> section add:
<clock> <timer name='hpet' present='yes'/> ...
Some tests require reasonable disk performance otherwise they time out (in particular the msi tests). For these we need to make sure to use QEMU's paravirtualized 'virtio' disk. The driver comes from KVM/Fedora: VirtIO drivers for Windows from Fedora.
QXL graphics driver
Some VMs use the QXL paravirtualized graphics card together with the driver from Spice's Windows Guest Tools pack.
- Mount spice-guest-tools-0.3.iso
- Then run spice-guest-tools-0.3.exe from the CD to install the driver.
- Windows will warn about each driver because they are unsigned but will still use any you install.
Also on Windows Vista Spice's Windows Guest Tools pack (0.3) does not install the QXL driver. It is possible to work around that by manually installing the driver directly from c:\Program Files\SPICE Guest Tools\drivers. One can either pick the Windows XP or Windows 7 one, they seem to behave identically. However the problem is that this immediately shows a very visible transparency/alpha blending issue in the Desktop Gadgets area (see (bug 61124). That does not seem to impact the tests though that configuration has a bunch of d3d8:device test failures like on Windows XP.
VMware graphics driver
QEMU can also emulate the paravirtualized VMware graphics card. However so far the VMware graphics driver has not shown to reduce the number failing tests or to improve the performance. Thus it is not used for now. Still here is how to install the driver on Windows 7 for further testing.
- These instructions are based on the I roughly followed 1 to comment in a discussion thread on Linux KVM.
- First go to VMware's site to download a VMware Tools ISO.
- Do not pick the latest VMware version because it only works with the VMware 7 hardware which QEMU 1.1.2 does not implement. The driver would install but then fail to load with error code 43 when it does not recognize the graphics card.
- Pick the ISO from the esx/4.1latest/windows directory which corresponds to VMware Tools 8.3.17.
- Mount the ISO and run 'setup.exe /a' and extract the drivers to c:\vmwaretools.
- Then right-click on the desktop background, and follow this sequence of buttons Screen Resolution -> Advanced Settings -> Properties -> Driver -> Update Driver -> Browse my computer.
- Pick the 'C:\vmwaretools\Common\VMware\Drivers\video' directory (or Common64 for the 64-bit drivers). This will install the 'VMware SVGA II' driver. This is in fact the legacy XPDM driver but it will still work in Windows 7.
- Do not pick the 'wddm_video' directory! It contains the same 'VMware SVGA 3D' that requires the VMware 7 hardware that is not supported by QEMU.
- Note that this procedure will not work in Windows 8 because it completely dropped support for the XPDM drivers. This means it is impossible to use the VMware drivers with QEMU on Windows 8.
The paravirtualized 'Virtio' network card causes errors (see Bug 1119281). Thus the VMs use either the e1000 or rtl8139 network cards with the standard Windows drivers.
- Installing unsigned drivers
- The Virtio drivers are unsigned and some Windows versions may require special steps to allow installing them.
- Windows 7 may need updates > SP1 before it will allow installing unsigned drivers. Then there are two options.
- Run gpedit.msc and in 'User Configuration' -> 'Administrative Templates' -> 'System' -> 'Driver Installation' look for 'Code Signing for Device Drivers' and set it to Enabled and either Warn or Ignore.
- Or on the command line run:
bcdedit.exe -set loadoptions DDISABLE_INTEGRITY_CHECKS bcdedit.exe -set TESTSIGNING ON # To undo bcdedit.exe -deletevalue DDISABLE_INTEGRITY_CHECKS bcdedit.exe -deletevalue TESTSIGNING
- Start Internet Explorer and answer the initial configuration questions.
- Rationale: It is not known if leaving these questions unanswered would modify the test results. However this would not correspond to any configuration that Windows applications would expect so in doubt it's best to answer them with something as close as possible to 'default' answers.
- Windows 10: Run iexplore.exe
- Disable the Internet Explorer automatic updates.
- Starting with version 10 Internet Explorer has its own independent update mechanism. The rationale for disabling it is the same as for Windows Update.
- Disable screensaver.
- Rationale: The VMs can remain idle for a long time so that the screensaver would possibly be running before before we start the tests. The screensaver could also trigger during the long WineTest.exe runs. In either case it seems better nor to risk having them interfere. Also we don't want half a dozen VMs to waste time on animated screensavers while waiting for a test to run.
- Windows 7: Control Panel -> Hardware and Sound -> Change when the computer sleeps
- Windows 10: Settings -> System -> Power & sleep
- Disable disk and computer suspend.
- Rationale: The VMs may remain idle for quite some time waiting for a test to run. We would not want them to 'suspend to RAM' as it's not clear they would successfully wake up when called upon. Similarly suspending the disk seems pointless and just asking for trouble.
- Disable Windows update.
- Rationale: After each test the VM is restored to a clean snapshot. So any work done by the Windows update process would be lost at that point. So the only effect it could have is messing the test environment and wasting computing resources.
- Windows 10: Disabling Windows Update can only be done by putting the network connection in metered mode.
- Early Windows 10: Note that Windows 10 1607 and older does not have a GUI to do this for an ethernet connection. But it can be done through the registry:
- [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\NetworkList\DefaultMediaCost]
- Right-Click on DefaultMediaCost -> Properties -> Advanced -> Change Owner -> Administrators -> OK
- Check "Replace owner on subcontainers and objects" -> OK
- Select Administrators -> Check "Full Control" -> OK
- Ethernet = dword:2.
- You can optionally change the DefaultMediaCost permissions and ownership to "NT Service\TrustedInstaller".
- Disable Delivery Optimization.
- Rationale: It seems like this service can use network bandwidth to exchange windows update data with the other Windows machines on the LAN even when Windows update is disabled. To do so, look for it near the Windows Update settings.
- Windows 10: Settings -> Update & Security -> Delivery Optimization
- Disable restore points.
- Rationale: Our MSI tests normally disable restore points and so should not need this. However restore points are redundant with the VM snapshots and just waste disk space.
- Windows 7/10: Run control -> Search Advanced -> View advanced system settings -> System Protection
- Reduce the boot delay.
- Rationale: When Windows crashes it waits for 30 seconds before rebooting. This may cause the Task to time out before the TestBot server has time to reconnect to the VM to retrieve the pre-crash test report.
- Run control -> Search Advanced -> View advanced system settings -> Startup and Recovery -> reduce the timeout to 5 seconds or so
- Disable search indexing.
- Rationale: Any indexing will have to be redone when the VM is reverted to the clean snapshot after the test. So all it can do is trash the disk, likely causing tests to time out.
- Windows 7/10: Run services.msc -> 'Windows Search' and switch it from 'Automatic (Delayed)' to 'Disabled'.
- Disable defragmentation.
- Rationale: Windows 10 regularly defragments the disk. Even though this is a low priority task it can slow down disk I/O and increase disk usage on the host.
- Windows 10: Right-click on the drive -> Properties -> Tools -> Optimize -> Disable Scheduled optimization.
- Disable OneDrive.
- Rationale: OneDrive would normally not be configured anyway and thus should not be using much resources. But it typically starts up on boot and can pop up Windows at inopportune times.
- Windows 10: Right-click on the OneDrive systray icon, and tell it not to run on startup.
- or (64-bit) run %SystemRoot%\SysWOW64\OneDriveSetup.exe /uninstall
- or (32-bit) run %SystemRoot%\System32\OneDriveSetup.exe /uninstall
- Disable time of last access.
- Rationale: Keeping track of the time of last access requires additional writes.
- Windows 10: fsutil behavior set disablelastaccess 1
- Disable Windows Defender.
- Rationale: Windows Defender sometimes mistakenly flags WineTest.exe as being infected. Also when restoring an old snapshot it may start a full disk scan which slows the tests.
- Windows 10: Set [HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows Defender] DisableAntiSpyware=dword:1
- Note that recent Windows Defender versions automatically reset this setting and re-enable themselves. This can be overridden through the Group Policy but only on Professional and Enterprise versions.
- Windows 10: Prevent Windows Defender from scanning files in the folder where WineTest is downloaded.
- Settings -> Update & Security -> Windows Security -> Virus & threat protection -> Manage settings -> Add or remove exclusions -> Add an exclusion
- Disable Windows Defender notifications.
- Rationale: If Windows Defender cannot be completely disabled (e.g. on the latest Windows 10), disable its most annoying notifications to avoid windows popping up during the user32:msg tests.
- Windows 10: Settings -> Update & Security -> Windows Security -> Open Windows Security Virus & threat protection -> Manage settings -> Change notification settings
- Hide the Windows 10 upgrade notifications.
- Rationale: Avoid windows popping up during the user32:msg tests.
- Windows 10: Setting -> System -> Notifications & actions -> Off + uncheck Windows experience, setup, tips & tricks
- Disable Windows Telemetry and the Customer Experience Improvement Program.
- One may want to avoid KB3068708, KB3075249, KB3080149. Also, in the Task Scheduler, disable the tasks in Microsoft > Windows > Application Experience and Customer Experience Improvement Program.
- Rationale: They involve tasks that get run on boot and cause a significant amount of disk activity.
- Activate Windows!
- Preferably once the hardware configuration is pretty much settled down to avoid having to reactivate again.
- Windows 10: Settings -> Update & Security -> Activation
- Lower the volume.
- Rationale: To preserve the ears of whoever is going to connect to the VM to perform maintenance.
- Reset TestAgentd's reboot count
- Last thing to do before the snapshot. Delete TestAgentd.exe.data.
The following configuration configuration options are not really important for the WineTestBot VMs but can help optimize real Windows partitions used for running WineTest.
- Configure auto-login.
- Rationale: This is necessary to get the tests to start automatically on boot.
- Windows 10: control userpasswords2 + uncheck "Users must enter a user name and password"
- or netplwiz + uncheck "Users must enter a user name and password"
- or [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon]
- DefaultUserName = string:<username>
- DefaultPassword = string:<password>
- AutoAdminLogon = string:"1"
- Disable hibernation.
- Rationale: This helps save a lot of space when taking the image of the Windows partition.
- Windows 10: powercfg -h off
- Disable swap.
- Rationale: This helps save a lot of space when taking the image of the Windows partition.
- Windows 7/10: Run control -> Search Advanced -> View advanced system settings -> Performance Settings -> Advanced
- Use 'dir /A:H c:\' to check that there is no pagefile.sys file.
See also VMware's recommendations to Optimize Windows Guest Operating System Performance.
Missing Windows dlls
Even with the latest Windows version and all updates some of the dlls Wine tests are missing. This section describes the recommended way to install them.
Note that atl80.dll is not found in c:\Windows\System32. Instead look for this set of dlls in c:\Windows\WinSxS\*.vc80.*, and optionally check the compatible versions in c:\Windows\WinSxS\Manifests\*.vc80.atl*.manifest.
This one is not tested yet, but for completeness it's best to install the x86/x64 Visual C++ 2008 SP1 redistributable. You may want to run Windows Update afterwards to get the latest fixes for this set of dlls.
Note that atl90.dll is not found in c:\Windows\System32. Instead look for this set of dlls in c:\Windows\WinSxS\*.vc90.*, and optionally check the compatible versions in c:\Windows\WinSxS\Manifests\*.vc90.atl*.manifest.
atl100, msvcp100, msvcr100
atl110, msvcp110, msvcr110
Install the x86 and x64 Visual C++ 2012 Update 4 redistributable. Note that it only supports Windows XP or greater. You may want to run Windows Update afterwards to get the latest fixes for this set of dlls.
Install the x86 and x64 Visual C++ 2013 redistributable. Note that it only supports Windows XP or greater. You may want to run Windows Update afterwards to get the latest fixes for this set of dlls.
msvcp140, msvcr140, ucrtbase
Install the x86 and x64 Visual C++ 2015-2019 redistributable. Note that it only supports Windows XP SP3 or greater. You may want to run Windows Update afterwards to get the latest fixes for this set of dlls.
It may look like this file is missing when running the 64-bit tests on a 64-bit version of Windows. This is because Direct3D 8 is deprecated and no 64-bit support for it is provided by Windows.
d3dcompiler_43, d3dx9_36, xinput1_3
WARNING: Uncheck the Bing Bar installation!
If these are missing on a 32-bit system install the optional Direct X components as documented above.
It may look like this file is missing when running the 64-bit tests on a 64-bit version of Windows. FIXME: Why? Is it obsolete like Direct3D 8?
dmband, dmime, dplayx
If these are missing on a 32-bit system install the optional Direct X components as documented above.
It may look like these files are missing when running the 64-bit tests on a 64-bit version of Windows. This is because DirectMusic and DirectPlay are deprecated and no 64-bit support for them is provided by Windows.
localspl, localui, spoolss
Wine implements the msxml6 dll but has no test for it yet. Still one can get MSXML6 SP1 here.
It may look like this file is missing when running the 32-bit tests on a 64-bit Windows system. This is because 64-bit Windows only has the 64-bit version of this file.
This is the lastest audio API for Windows 7+. It may be installed through this download.
- Use the fvwm window manager
- Rationale: Most window managers are buggy, in particular the GNOME and KDE ones, and cause some tests to fail or even crash. fvwm is the one known 'good' windows manager.
- Use a Windows-compatible fvwm configuration
- Rationale: Some tests verify that Windows applications get the right sequence of messages when windows lose/gain focus, etc. The sequence of messages depends in part on the window manager and cannot be right if the Unix window manager does not behave like Windows does, particularly wrt. focus-follows-mouse, transient window decorations, etc.
# The default fvwm config is pretty long and has tons of features of # dubious relevance for a test environment: # - It has a sidebar which may be causing the ole32:dragdrop test to fail. # - The virtual desktops are just an annoyance when accessing the machine # through VNC. # So this configuration file completely replaces the default one and is # built from scratch with only the settings that have been deemed necessary. # fvwm's default SloppyFocus policy does not match the Windows behavior and # causes a lot of failures. So use ClickToFocus instead which best matches the # Windows behavior. Style * ClickToFocus # DontStackTransientParent is needed to avoid some failures in user32:win. # See https://www.winehq.org/pipermail/wine-devel/2018-February/122021.html Style * DontStackTransientParent # Wine uses MWM hints to get the proper window behavior. # So tell fvwm to obey these hints. Style * MwmDecor # DecorateTransient does not seem to be needed for the tests but gives a proper # border to transient Windows like File Open dialogs. This makes them much more # usable and better matches the Windows behavior. Style * DecorateTransient # Don't use virtual desktops to avoid aggravation when the mouse goes out of # the VNC window. DesktopSize 1x1
- Install ttf-mscorefonts-installer (non-free) or fonts-liberation
- Rationale: No Windows system would ship without a base set of fonts of various families (Serif, Sans Serif). Since the lack of such fonts is not a valid Windows configuration, it is reasonable for tests (e.g. gdiplus:font) to depend on them and fail if they are missing. Furthermore ttf-mscorefonts-installer is preferred because it provides Arial which some tests need (but they skip if Arial is missing).