Wine Developer's Guide/Wine and Multimedia

From WineHQ Wiki
Jump to: navigation, search

This file contains information about the implementation of the multimedia layer of Wine.

The implementation can be mainly found in a number of directories:

  • dlls/mmdevapi, dlls/dsound and dlls/winmm for the main sound APIs.
  • dlls/msacm32 (for the audio compression/decompression manager)
  • dlls/msvfw32 (for the video compression/decompression manager).


1 Overview

  • the high level abstraction layer: mmdevapi, dsound and winmm DLLs.
  • the mid level layer: handles MCI commands, providing coarser grain operations (like playing a MIDI file, or playing a video stream).
  • the low level layer: device drivers, currently implemented in winealsa.drv, winecoreaudio.drv, and wineoss.drv DLLs. It also has some helper DLLs (like msacm32 and msvfw32), which can make use of external DLLs like mciavi32 or application-provided DLLs.

The low level layer may depend on current hardware and OS services (like OSS on Unix). It provides the core of playback/record using fine grain objects (audio/midi streams...).

The dsound and winmm legacy APIs are implemented on top of mmdevapi, the new Windows audio system introduced with Windows Vista (Multimedia Device API).

2 Multimedia architecture

2.1 Windows 95 multimedia architecture

             |
Kernel space |                    Client applications
             |
             |           | |         ^ ^       | |          | |
             |        16>| |<32   16>| |<32 16>| |<32    16>| |<32
             |           | v         | |       | v          | v
             |      +----|-----------|---------|------------|-------+
             |      |    |           |         |            |       |  WinMM.dll
             |      |    |           |         |            |       |   32-bit
             |      +----|-----------|---------|------------|-------+
             |           | |         | ^       | |          |
             |  +------+ | |<16      | |       | |<16       |
             |  |   16>| | |         | |       | |          |
             |  |      v v v         | |       v v          v
             |  |   +---------------+---+-------------+-------------+
             |  |   | waveInXXX     |   | mciXXX      | *playSound* |
             |  |   | waveOutXXX    |   |             | mmioXXX     |
             |  |   | midiInXXX     |   |             | timeXXX     |
             |  |   | midiOutXXX    |   |             | driverXXX   |
             |  |   | midiStreamXXX |   |             |             |  MMSystem.dll
             |  |   | mixerXXX      |   |             |             |     16-bit
 +--------+  |  |   | auxXXX    +---+   +---+ mmThread|             |
 |MMDEVLDR|<------->| joyXXX    | Call back | mmTask  |             |
 +--------+  |  |   +-----------+-----------+---------+-------------+
     ^       |  |          |      ^    ^       | ^
     |       |  |       16>|      |<16>|    16>| |<16
     v       |  |          v      |    |       v |
 +--------+  |  |   +-------------+    +----------+
 |  VxD   |<------->|    *.drv    |    | mci*.drv |
 +--------+  |  |   +--------------+   +-----------+
             |  |    |  msacm.drv  |    | mciwave  |
             |  |    +--------------+   +-----------+
             |  |     | midimap.drv |    | mcimidi  |
             |  |     +-------------+    +-----------+
             |  |    Low-level drivers    |    ...   | MCI drivers
             |  |                         +----------+
             |  |                               |
             |  |                               |<16
             |  +-------------------------------+
             |

The important points to notice are:

  • all drivers (and most of the core code) is 16-bit
  • all hardware (or most of it) dependent code reside in the kernel space (which is not surprising)

2.2 Windows NT multimedia architecture

Note that Win 98 has mixed 95/NT architecture, so when speaking about Windows 95 (resp. NT) architecture, it refers to the type of architecture, not what's actually implemented. For example, Windows 98 implements both types of architectures.

The important points to notice (compared to the Windows 95 architecture) are:

  • drivers (low level, MCIs...) are 32-bit and Unicode
  • the interfaces between kernel and user drivers has changed, but it doesn't impact much Wine. Those changes allow some good things (like kernel mixing, where different apps share the audio hardware) and of course bad things (like kernel mixing, which adds latency).

2.3 Wine multimedia architecture

             |
Kernel space |                    Client applications
             |
             |           | |         ^ ^       | |          | |
             |        16>| |<32   16>| |<32 16>| |<32    16>| |<32
             |           | |         | |       | |          | |
             |  +------+ | |         | |       | |          | |
             |  |32/16>| | |         | |       | |          | |
             |  |      v v v         | |       v v          v v
             |  |   +---------------+---+-------------+-------------+
             |  |   | waveInXXX     |   | mciXXX      | *playSound* |
             |  |   | waveOutXXX    |   |             | mmioXXX     | winmm.dll
             |  |   | midiInXXX     |   |             | timeXXX     |   32-bit
             |  |   | midiOutXXX    |   |             | driverXXX   |
             |  |   | midiStreamXXX |   |             |             | mmsystem.dll16
             |  |   | mixerXXX      |   |             |             |   16-bit
             |  |   | auxXXX    +---+   +---+ mmThread|             |
             |  |   | joyXXX    | Call back | mmTask  |             |
             |  |   +-----------+-----------+---------+-------------+
             |  |         ||      ^    ^       | ^
             |  |      16>||<32   |<16>|    16>| |<16
             |  |         vv      |<32>|    32>v |<32
+---------+  |  |   +-------------+    +----------+
|HW driver|<------->|    *.drv    |    | mci*.drv |
+---------+  |  |   +--------------+   +-----------+
             |  |    | msacm32.drv |    | mciwave  |
             |  |    +--------------+   +-----------+
             |  |     | midimap.drv |    | mcimidi  |
             |  |     +-------------+    +-----------+
             |  |    Low-level drivers    |    ...   | MCI drivers
             |  |                         +----------+
             |  |                               |
             |  |                               |<32/16
             |  +-------------------------------+
             |

From the previous drawings, the most noticeable differences are:

  • low-level drivers can either be 16- or 32-bit (in fact, Wine supports only native wave and audio mappers).
  • MCI drivers can either be 16- or 32-bit
  • all built-in drivers (low-level and MCI) will be written as 32-bit drivers

Wine WinMM automatically adapts the messages to be sent to a driver so that it can convert it to 16 or 32-bit interfaces.

3 Low level layers

The low level drivers abstract the hardware specific features from the rest of the multimedia code. Those are implemented on top of mmdevapi, as new versions of Windows do.

Please note that native low level drivers are not currently supported in Wine, because they either access hardware components or require VxDs to be loaded; Wine does not correctly supports those two so far.

There are two specific low level drivers (msacm32.drv for wave input/output, midimap.drv for MIDI output only). These drivers (also present in Windows):

  • allow to choose one low level driver between many (we'll discuss how the choice is made later on)
  • add the possibility to convert stream format (i.e. ADPCM => PCM) (this is useful if the format required by the application for playback isn't supported by the soundcard).
  • add the possibility to filter a stream (adding echo, equalizer... to a wave stream, or modify the instruments that have to be played for a MIDI stream).

3.1 Hardware-bound low level drivers

Wine currently supports the following (kernel) multimedia interfaces.

  • Open Sound System (OSS) as supplied in the Linux and FreeBSD kernels by 4Front Technologies. The presence of this driver is checked by configure (depends on the <sys/soundcard.h> file). Source code resides in dlls/wineoss.drv.
  • Advanced Linux Sound Architecture (ALSA) as supplied in the Linux kernel. Source code resides in dlls/winealsa.drv.
  • Core Audio as introduced in Mac OS X 10.3. Source code resides in dlls/winecoreaudio.drv.

3.2 Wave mapper (msacm32.drv)

The Wave mapper device allows to load on-demand audio codecs in order to perform software conversion for the types the actual low level driver (hardware). Those codecs are provided through the standard ACM drivers in msacm32 DLL.

Wave mapper driver implementation can be found in dlls/msacm32.drv directory. This driver heavily relies on msacm DLL found in dlls/msacm32. This DLL loads ACM drivers which provide the conversion to PCM format (which is normally supported by low level drivers). A-Law, μLaw, ADPCM, MP3... fit into the category of non PCM formats.

3.3 MIDI mapper (midimap)

Midi mapper allows to map each one of 16 MIDI channels to a specific instrument on an installed sound card. This allows for example to support different MIDI instrument definitions (XM, GM...). It also permits to output on a per channel basis to different MIDI renderers.

A built-in MIDI mapper can be found in dlls/midimap. It partly provides the same functionality as the Windows one. It allows to pick up destination channels: you can map a given channel to a specific playback device channel (see the configuration bits for more details).

4 Mid level drivers (MCI)

The mid level drivers are represented by some common API functions, mostly mciSendCommand and mciSendString. Wine implements several MCI mid level drivers.

Table 13-1. Wine MCI drivers

MCI Name DLL Name Role Location Comments
CdAudio mcicda MCI interface to a CD audio player dlls/mcicda Relies on NTDLL CD-ROM raw interface (through DeviceIoControl).
WaveAudio mciwave MCI interface for wave playback and record dlls/mciwave It uses the low level audio API.
Sequencer mciseq Midi Sequencer (playback) dlls/mciseq It uses the low level midi APIs
AviVideo mciavi32 AVI playback and record dlls/mciavi32 It rather heavily relies on msvfw32 DLL to work.

The MCI Name column is the name of the MCI driver, as it is searched in configuration. The DLL Name column is the name of the DLL the configuration provides as a value. The name listed here is the default one (see the configuration section for the details).

Adding a new MCI driver is just a matter of writing the corresponding DLL with the correct interface (see existing MCI drivers for the details), and to provide the relevant setup information for wine.inf.

5 High level layers

5.1 winmm

The high level layers encompass basically the winmm DLL exported APIs. It also provides the skeleton for the core functionality for multimedia playback and recording. Note that native WINMM and MMSYSTEM do not currently work under Wine and there is no plan to support them (it would require to also fully support VxD, which is not done yet).

winmm and mmsystem.dll16 in Wine can handle both 32- and 16-bit drivers (for low level and MCI drivers). Wine will handle all the conversions transparently for all the calls to winmm and mmsystem.dll16, as it knows what the driver interface is (32- or 16-bit) and it manages the information sent accordingly.

MCI drivers are seen as regular Wine modules, and can be loaded (with a correct load order between builtin, native), as any other DLL.

Multimedia timers are implemented with a dedicated thread, run in the context of the calling process, which should correctly mimic Windows behavior. The only drawback is that the thread will appear the calling process if it enumerates the running processes.

5.2 dsound

Wine also provide a dsound (DirectX) DLL with the proper COM implementation.

6 MS ACM DLLs

6.1 Contents

The msacm32 DLL provides a way to map a given wave format to another format. It also provides filtering capabilities. Those DLLs only implement the proper switch between a caller and a driver providing the implementation of the requested format change or filter operation.

There's nothing specific in Wine implementation compared to Windows one. Here's however a list of the builtin format change drivers (there's no filter driver yet):

Table 13-2. Wine ACM drivers

Name Provides
imaadp32 IMA ADPCM (adaptative PCM)
msadp32 Microsoft ADPCM (adaptative PCM)
msg711 Microsoft G.711 (A-Law and μ-Law)
msgsm32 Microsoft GSM 06.10
winemp3 Wine MP3 (MPEG Layer 3), based on mpglib library

Note that Wine also supports native audio codecs as well.

All builtin ACM drivers are 32-bit Unicode DLLs.

6.2 Caching

msacm32 keeps some data cached for all known ACM drivers. Under the key

Software\Microsoft\AudioCompressionManager\DriverCache\driver_name

are kept for values:

  • aFormatTagCache which contains an array of DWORD. There are two DWORDs per cFormatTags entry. The first DWORD contains a format tag value, and the second the associated maximum size for a WAVEFORMATEX structure. (Fields dwFormatTag and cbFormatSize from ACMFORMATDETAILS)
  • cFilterTags contains the number of tags supported by the driver for filtering.
  • cFormatTags contains the number of tags support by the driver for conversions.
  • fdwSupport (the same as the one returned from acmDriverDetails).

The cFilterTags, cFormatTags, fdwSupport are the same values as the ones returned from acmDriverDetails function.

7 MS Video DLLs

7.1 Contents

msvfw32 provides encode/decode video streams. Those DLLs only implement the proper switch between a caller and a driver providing the implementation of the requested format coding/decoding operation.

There's nothing specific in Wine implementation compared to Windows one. Here's however a list of the builtin decoding drivers:

Table 13-3. Wine VIDC drivers

Name Provides
msrle32 Microsoft RLE (Run-Length encoded)
msvidc32 Microsoft Video-1
iccvid Radius Cinepak Video Decoder

Note that Wine also supports native video codecs as well.

All builtin VIDC drivers are 32-bit Unicode DLLs.

8 Multimedia configuration

Unfortunately, multimedia configuration evolved over time:

  • In the early days on Windows 3.x, configuration was stored in system.ini file, under various sections ([drivers] for low level drivers, [mci] (resp. [mci32]) for 16-bit (resp. 32-bit) MCI drivers...).

  • With the apparition of the registry, in Windows 95, configuration as been duplicated there, under the key

    HKLM\System\CurrentControlSet\Control\MediaResources
  • Windows NT also adopted the registry, but decided to store the configuration information under another key than Windows 9x did.

    HKLM\Software\Microsoft\Windows NT\CurrentVersion

    And with a different layout of keys and values beneath this key.

Currently, Wine tries to load first a driver (low-level or MCI) from the NT registry settings. If it fails, it will try the system.ini configuration.

An out-of-the-box configuration is provided in wine.inf, and shall be stored in registry and system.ini at Wine installation time. It will setup correctly the MCI drivers configuration (as well as the wave and MIDI mappers). As the low-level drivers depend on hardware, their setup will be handled by winecfg.

Table 13-4. Wine multimedia configuration scheme

Driver Read from NT registry Read from system.ini Setup by wine.inf Setup by winecfg
MCI drivers Yes (1) Yes (2) Yes No
Wave and MIDI mappers Yes No Yes No
Hardware-bound low level drivers Yes No No Yes
ACM and VIDC drivers (audio & video codecs) No Yes Yes No

This will allow most settings to be correctly loaded and handled. However, it won't if an app tries to search directly the registry for the actual configuration, as the three potential configuration places may not be in sync.

It still lacks a correct installation scheme (as any multimedia device under Windows), so that all the correct keys are created in the registry. This requires an advanced model since, for example, the number of wave out devices can only be known on the destination system (depends on the sound card driven by the OSS interface).

The following sections describe which type of information (depending on the location) Wine multimedia DLLs understand.

8.1 NT configuration

Under the

HKLM\Software\Microsoft\Windows NT\CurrentVersion

key, are stored the names of the DLLs to be loaded for each MCI driver name:

"cdaudio"="mcicda.drv"
"sequencer"="mciseq.drv"
"waveaudio"="mciwave.drv"
"avivideo"="mciavi.drv"
"videodisc"="mcipionr.drv"
"vcr"="mcivisca.drv"
"MPEGVideo"="mciqtz.drv"

8.2 system.ini

Wine will read the MCI drivers from the [mci] or [mci32] section. Wine won't make any difference between the two.

Here's a sample configuration:

[mci]
MPEGVideo=mciqtz32.dll
MPEGVideo2=mciqtz32.dll
avivideo=mciavi32.dll
cdaudio=mcicda.dll
sequencer=mciseq.dll
vcr=mcivisca.drv
waveaudio=mciwave.dll

ACM drivers configuration is read (only so far) from the system.ini (and setup at Wine installation from the wine.inf file).

[drivers32]
msacm.imaadpcm=imaadp32.acm
msacm.msadpcm=msadp32.acm
msacm.msg711=msg711.acm
msacm.winemp3=winemp3.acm
msacm.msgsm610=msgsm32.acm

Video (aka vidc) drivers configuration is read (only so far) from the system.ini (and setup at Wine installation from the wine.inf file).

[drivers32]
vidc.mrle=msrle32.dll
vidc.msvc=msvidc32.dll
vidc.cvid=iccvid.dll

See also the configuration part of the User's Guide for other information on low level drivers.

8.3 Per driver/DLL configuration

8.3.1 Midi mapper

The Midi mapper configuration is the same as on Windows 9x. Under the key:

HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Multimedia\MIDIMap

if the UseScheme value is not set, or is set to a null value, the MIDI mapper will always use the driver identified by the CurrentInstrument value. Note: Wine (for simplicity while installing) allows to define CurrentInstrument as #n (where n is a number), whereas Windows only allows the real device name here. If UseScheme is set to a non null value, CurrentScheme defines the name of the scheme to map the different channels. All the schemes are available with keys like

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\MediaProperties\PrivateProperties\Midi\Schemes\scheme_name

For every scheme, under this key, will be a sub-key (which name is usually a two digit index, starting at 00). Its default value is the name of the output driver, and the value Channels lists all channels (of the 16 standard MIDI ones) which have to be copied to this driver.

To provide enhanced configuration and mapping capabilities, each driver can define under the key

HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\MediaProperties\PrivateProperties\Midi\Ports\driver_name

a link to and .IDF file which allows to remap channels internally (for example 9 -> 16), to change instruments identification, event controllers values. See dlls/midimap/midimap.c for the details (this isn't implemented yet).