Difference between revisions of "Wine Developer's Guide/Wine and Multimedia"
(→Contents: fix markup)
(→Caching: fix markup)
|Line 250:||Line 250:|
=== Caching ===
=== Caching ===
<>msacm32</> keeps some data cached for all known ACM drivers. Under the key
are kept for values:
are kept for values:
* <>aFormatTagCache</> which contains an array of <>DWORD</>. There are two <>DWORD</>s per <>cFormatTags</> entry. The first <>DWORD</> contains a format tag value, and the second the associated maximum size for a WAVEFORMATEXstructure. (Fields dwFormatTagand cbFormatSizefrom ACMFORMATDETAILS)
* <>cFilterTags</> contains the number of tags supported by the driver for filtering.
* <>cFormatTags</> contains the number of tags support by the driver for conversions.
* <>fdwSupport</> (the same as the one returned from <code>acmDriverDetails</code>).
The <>cFilterTags</>, <>cFormatTags</>, <>fdwSupport</> are the same values as the ones returned from <code>acmDriverDetails</code> function.
== MS Video DLLs ==
== MS Video DLLs ==
Revision as of 17:16, 7 February 2016
This file contains information about the implementation of the multimedia layer of Wine.
The implementation can be mainly found in a number of directories:
- dlls/mmdevapi, dlls/dsound and dlls/winmm for the main sound APIs.
- dlls/msacm32 (for the audio compression/decompression manager)
- dlls/msvfw32 (for the video compression/decompression manager).
- 1 Overview
- 2 Multimedia architecture
- 3 Low level layers
- 4 Mid level drivers (MCI)
- 5 High level layers
- 6 MS ACM DLLs
- 7 MS Video DLLs
- 8 Multimedia configuration
- the high level abstraction layer: mmdevapi, dsound and winmm DLLs.
- the mid level layer: handles MCI commands, providing coarser grain operations (like playing a MIDI file, or playing a video stream).
- the low level layer: device drivers, currently implemented in winealsa.drv, winecoreaudio.drv, and wineoss.drv DLLs. It also has some helper DLLs (like msacm32 and msvfw32), which can make use of external DLLs like mciavi32 or application-provided DLLs.
The low level layer may depend on current hardware and OS services (like OSS on Unix). It provides the core of playback/record using fine grain objects (audio/midi streams...).
The dsound and winmm legacy APIs are implemented on top of mmdevapi, the new Windows audio system introduced with Windows Vista (Multimedia Device API).
2 Multimedia architecture
2.1 Windows 95 multimedia architecture
| Kernel space | Client applications | | | | ^ ^ | | | | | 16>| |<32 16>| |<32 16>| |<32 16>| |<32 | | v | | | v | v | +----|-----------|---------|------------|-------+ | | | | | | | WinMM.dll | | | | | | | 32-bit | +----|-----------|---------|------------|-------+ | | | | ^ | | | | +------+ | |<16 | | | |<16 | | | 16>| | | | | | | | | | v v v | | v v v | | +---------------+---+-------------+-------------+ | | | waveInXXX | | mciXXX | *playSound* | | | | waveOutXXX | | | mmioXXX | | | | midiInXXX | | | timeXXX | | | | midiOutXXX | | | driverXXX | | | | midiStreamXXX | | | | MMSystem.dll | | | mixerXXX | | | | 16-bit +--------+ | | | auxXXX +---+ +---+ mmThread| | |MMDEVLDR|<------->| joyXXX | Call back | mmTask | | +--------+ | | +-----------+-----------+---------+-------------+ ^ | | | ^ ^ | ^ | | | 16>| |<16>| 16>| |<16 v | | v | | v | +--------+ | | +-------------+ +----------+ | VxD |<------->| *.drv | | mci*.drv | +--------+ | | +--------------+ +-----------+ | | | msacm.drv | | mciwave | | | +--------------+ +-----------+ | | | midimap.drv | | mcimidi | | | +-------------+ +-----------+ | | Low-level drivers | ... | MCI drivers | | +----------+ | | | | | |<16 | +-------------------------------+ |
The important points to notice are:
- all drivers (and most of the core code) is 16-bit
- all hardware (or most of it) dependent code reside in the kernel space (which is not surprising)
2.2 Windows NT multimedia architecture
Note that Win 98 has mixed 95/NT architecture, so when speaking about Windows 95 (resp. NT) architecture, it refers to the type of architecture, not what's actually implemented. For example, Windows 98 implements both types of architectures.
The important points to notice (compared to the Windows 95 architecture) are:
- drivers (low level, MCIs...) are 32-bit and Unicode
- the interfaces between kernel and user drivers has changed, but it doesn't impact much Wine. Those changes allow some good things (like kernel mixing, where different apps share the audio hardware) and of course bad things (like kernel mixing, which adds latency).
2.3 Wine multimedia architecture
| Kernel space | Client applications | | | | ^ ^ | | | | | 16>| |<32 16>| |<32 16>| |<32 16>| |<32 | | | | | | | | | | +------+ | | | | | | | | | |32/16>| | | | | | | | | | | v v v | | v v v v | | +---------------+---+-------------+-------------+ | | | waveInXXX | | mciXXX | *playSound* | | | | waveOutXXX | | | mmioXXX | winmm.dll | | | midiInXXX | | | timeXXX | 32-bit | | | midiOutXXX | | | driverXXX | | | | midiStreamXXX | | | | mmsystem.dll16 | | | mixerXXX | | | | 16-bit | | | auxXXX +---+ +---+ mmThread| | | | | joyXXX | Call back | mmTask | | | | +-----------+-----------+---------+-------------+ | | || ^ ^ | ^ | | 16>||<32 |<16>| 16>| |<16 | | vv |<32>| 32>v |<32 +---------+ | | +-------------+ +----------+ |HW driver|<------->| *.drv | | mci*.drv | +---------+ | | +--------------+ +-----------+ | | | msacm32.drv | | mciwave | | | +--------------+ +-----------+ | | | midimap.drv | | mcimidi | | | +-------------+ +-----------+ | | Low-level drivers | ... | MCI drivers | | +----------+ | | | | | |<32/16 | +-------------------------------+ |
From the previous drawings, the most noticeable differences are:
- low-level drivers can either be 16- or 32-bit (in fact, Wine supports only native wave and audio mappers).
- MCI drivers can either be 16- or 32-bit
- all built-in drivers (low-level and MCI) will be written as 32-bit drivers
Wine WinMM automatically adapts the messages to be sent to a driver so that it can convert it to 16 or 32-bit interfaces.
3 Low level layers
The low level drivers abstract the hardware specific features from the rest of the multimedia code. Those are implemented on top of mmdevapi, as new versions of Windows do.
Please note that native low level drivers are not currently supported in Wine, because they either access hardware components or require VxDs to be loaded; Wine does not correctly supports those two so far.
There are two specific low level drivers (msacm32.drv for wave input/output, midimap.drv for MIDI output only). These drivers (also present in Windows):
- allow to choose one low level driver between many (we'll discuss how the choice is made later on)
- add the possibility to convert stream format (i.e. ADPCM => PCM) (this is useful if the format required by the application for playback isn't supported by the soundcard).
- add the possibility to filter a stream (adding echo, equalizer... to a wave stream, or modify the instruments that have to be played for a MIDI stream).
3.1 Hardware-bound low level drivers
Wine currently supports the following (kernel) multimedia interfaces.
- Open Sound System (OSS) as supplied in the Linux and FreeBSD kernels by 4Front Technologies. The presence of this driver is checked by configure (depends on the <sys/soundcard.h> file). Source code resides in dlls/wineoss.drv.
- Advanced Linux Sound Architecture (ALSA) as supplied in the Linux kernel. Source code resides in dlls/winealsa.drv.
- Core Audio as introduced in Mac OS X 10.3. Source code resides in dlls/winecoreaudio.drv.
3.2 Wave mapper (msacm32.drv)
The Wave mapper device allows to load on-demand audio codecs in order to perform software conversion for the types the actual low level driver (hardware). Those codecs are provided through the standard ACM drivers in msacm32 DLL.
Wave mapper driver implementation can be found in dlls/msacm32.drv directory. This driver heavily relies on msacm DLL found in dlls/msacm32. This DLL loads ACM drivers which provide the conversion to PCM format (which is normally supported by low level drivers). A-Law, μLaw, ADPCM, MP3... fit into the category of non PCM formats.
3.3 MIDI mapper (midimap)
Midi mapper allows to map each one of 16 MIDI channels to a specific instrument on an installed sound card. This allows for example to support different MIDI instrument definitions (XM, GM...). It also permits to output on a per channel basis to different MIDI renderers.
A built-in MIDI mapper can be found in dlls/midimap. It partly provides the same functionality as the Windows one. It allows to pick up destination channels: you can map a given channel to a specific playback device channel (see the configuration bits for more details).
4 Mid level drivers (MCI)
The mid level drivers are represented by some common API functions, mostly
mciSendString. Wine implements several MCI mid level drivers.
|MCI Name||DLL Name||Role||Location||Comments|
|CdAudio||mcicda||MCI interface to a CD audio player||dlls/mcicda|| Relies on NTDLL CD-ROM raw interface (through |
|WaveAudio||mciwave||MCI interface for wave playback and record||dlls/mciwave||It uses the low level audio API.|
|Sequencer||mciseq||Midi Sequencer (playback)||dlls/mciseq||It uses the low level midi APIs|
|AviVideo||mciavi32||AVI playback and record||dlls/mciavi32||It rather heavily relies on msvfw32 DLL to work.|
The MCI Name column is the name of the MCI driver, as it is searched in configuration. The DLL Name column is the name of the DLL the configuration provides as a value. The name listed here is the default one (see the configuration section for the details).
Adding a new MCI driver is just a matter of writing the corresponding DLL with the correct interface (see existing MCI drivers for the details), and to provide the relevant setup information for wine.inf.
5 High level layers
The high level layers encompass basically the winmm DLL exported APIs. It also provides the skeleton for the core functionality for multimedia playback and recording. Note that native WINMM and MMSYSTEM do not currently work under Wine and there is no plan to support them (it would require to also fully support VxD, which is not done yet).
winmm and mmsystem.dll16 in Wine can handle both 32- and 16-bit drivers (for low level and MCI drivers). Wine will handle all the conversions transparently for all the calls to winmm and mmsystem.dll16, as it knows what the driver interface is (32- or 16-bit) and it manages the information sent accordingly.
MCI drivers are seen as regular Wine modules, and can be loaded (with a correct load order between builtin, native), as any other DLL.
Multimedia timers are implemented with a dedicated thread, run in the context of the calling process, which should correctly mimic Windows behavior. The only drawback is that the thread will appear the calling process if it enumerates the running processes.
Wine also provide a dsound (DirectX) DLL with the proper COM implementation.
6 MS ACM DLLs
The msacm32 DLL provides a way to map a given wave format to another format. It also provides filtering capabilities. Those DLLs only implement the proper switch between a caller and a driver providing the implementation of the requested format change or filter operation.
There's nothing specific in Wine implementation compared to Windows one. Here's however a list of the builtin format change drivers (there's no filter driver yet):
|imaadp32||IMA ADPCM (adaptative PCM)|
|msadp32||Microsoft ADPCM (adaptative PCM)|
|msg711||Microsoft G.711 (A-Law and μ-Law)|
|msgsm32||Microsoft GSM 06.10|
|winemp3||Wine MP3 (MPEG Layer 3), based on mpglib library|
Note that Wine also supports native audio codecs as well.
All builtin ACM drivers are 32-bit Unicode DLLs.
msacm32 keeps some data cached for all known ACM drivers. Under the key
are kept for values:
- aFormatTagCache which contains an array of DWORD. There are two DWORDs per cFormatTags entry. The first DWORD contains a format tag value, and the second the associated maximum size for a
- cFilterTags contains the number of tags supported by the driver for filtering.
- cFormatTags contains the number of tags support by the driver for conversions.
- fdwSupport (the same as the one returned from
The cFilterTags, cFormatTags, fdwSupport are the same values as the ones returned from
7 MS Video DLLs
msvfw32 provides encode/decode video streams. Those DLLs only implement the proper switch between a caller and a driver providing the implementation of the requested format coding/decoding operation.
There's nothing specific in Wine implementation compared to Windows one. Here's however a list of the builtin decoding drivers:
||Microsoft RLE (Run-Length encoded)|
||Radius Cinepak Video Decoder|
Note that Wine also supports native video codecs as well.
All builtin VIDC drivers are 32-bit Unicode DLLs.
8 Multimedia configuration
Unfortunately, multimedia configuration evolved over time:
In the early days on Windows 3.x, configuration was stored in
system.inifile, under various sections (
[drivers]for low level drivers,
[mci32]) for 16-bit (resp. 32-bit) MCI drivers...).
With the apparition of the registry, in Windows 95, configuration as been duplicated there, under the key
Windows NT also adopted the registry, but decided to store the configuration information under another key than Windows 9x did.
And with a different layout of keys and values beneath this key.
Currently, Wine tries to load first a driver (low-level or MCI) from the NT registry settings. If it fails, it will try the
An out-of-the-box configuration is provided in
wine.inf, and shall be stored in registry and
system.ini at Wine installation time. It will setup correctly the MCI drivers configuration (as well as the wave and MIDI mappers). As the low-level drivers depend on hardware, their setup will be handled by
|Driver||Read from NT registry|| Read from
|| Setup by
|| Setup by |
|MCI drivers||Yes (1)||Yes (2)||Yes||No|
|Wave and MIDI mappers||Yes||No||Yes||No|
|Hardware-bound low level drivers||Yes||No||No||Yes|
|ACM and VIDC drivers (audio & video codecs)||No||Yes||Yes||No|
This will allow most settings to be correctly loaded and handled. However, it won't if an app tries to search directly the registry for the actual configuration, as the three potential configuration places may not be in sync.
It still lacks a correct installation scheme (as any multimedia device under Windows), so that all the correct keys are created in the registry. This requires an advanced model since, for example, the number of wave out devices can only be known on the destination system (depends on the sound card driven by the OSS interface).
The following sections describe which type of information (depending on the location) Wine multimedia DLLs understand.
8.1 NT configuration
key, are stored the names of the DLLs to be loaded for each MCI driver name:
"cdaudio"="mcicda.drv" "sequencer"="mciseq.drv" "waveaudio"="mciwave.drv" "avivideo"="mciavi.drv" "videodisc"="mcipionr.drv" "vcr"="mcivisca.drv" "MPEGVideo"="mciqtz.drv"
Wine will read the MCI drivers from the
[mci32] section. Wine won't make any difference between the two.
Here's a sample configuration:
[mci] MPEGVideo=mciqtz32.dll MPEGVideo2=mciqtz32.dll avivideo=mciavi32.dll cdaudio=mcicda.dll sequencer=mciseq.dll vcr=mcivisca.drv waveaudio=mciwave.dll
ACM drivers configuration is read (only so far) from the
system.ini (and setup at Wine installation from the
[drivers32] msacm.imaadpcm=imaadp32.acm msacm.msadpcm=msadp32.acm msacm.msg711=msg711.acm msacm.winemp3=winemp3.acm msacm.msgsm610=msgsm32.acm
vidc) drivers configuration is read (only so far) from the
system.ini (and setup at Wine installation from the
[drivers32] vidc.mrle=msrle32.dll vidc.msvc=msvidc32.dll vidc.cvid=iccvid.dll
See also the configuration part of the User's Guide for other information on low level drivers.
8.3 Per driver/DLL configuration
8.3.1 Midi mapper
The Midi mapper configuration is the same as on Windows 9x. Under the key:
UseScheme value is not set, or is set to a null value, the MIDI mapper will always use the driver identified by the
CurrentInstrument value. Note: Wine (for simplicity while installing) allows to define
#n (where n is a number), whereas Windows only allows the real device name here. If
UseScheme is set to a non null value,
CurrentScheme defines the name of the scheme to map the different channels. All the schemes are available with keys like
For every scheme, under this key, will be a sub-key (which name is usually a two digit index, starting at 00). Its default value is the name of the output driver, and the value
Channels lists all channels (of the 16 standard MIDI ones) which have to be copied to this driver.
To provide enhanced configuration and mapping capabilities, each driver can define under the key
a link to and
.IDF file which allows to remap channels internally (for example 9 -> 16), to change instruments identification, event controllers values. See
dlls/midimap/midimap.c for the details (this isn't implemented yet).