Using WAVEFORMATEXTENSIBLE with WAVE_FORMAT_IEEE_FLOAT, waveOutOpen returns WAVERR_BADFORMAT

Question

I have been using WAVEFORMATEX with WaveOut to play audio in Windows, in stereo at rates from 44.1KHz to 192KHz using WAVE_FORMAT_IEEE_FLOAT. The program is written in C++ and compiled in MinGW. This is all working correctly:

https://github.com/Raptor007/AutoDJ/blob/60f4debca2103e669a5d1b822b04c73cdcdaf05b/AutoDJ.cpp#L2412-L2435

Now I'm trying to expand to quadraphonic multi-channel output, which seems to require WAVEFORMATEXTENSIBLE, a superset of WAVEFORMATEX. Here is the relevant code with those changes applied:

WAVEFORMATEXTENSIBLE wfx;
memset( &wfx, 0, sizeof(wfx) );
wfx.Format.nChannels = want.channels;
wfx.dwChannelMask = (want.channels == 4) ? 0x33 : ((want.channels == 1) ? 0x4 : 0x3);
wfx.Format.nSamplesPerSec = want.freq;
wfx.Format.cbSize = sizeof(wfx) - sizeof(wfx.Format);
MMRESULT wave_out_result = ~MMSYSERR_NOERROR;
if( userdata.HighRes )
{
    wfx.Format.wFormatTag = WAVE_FORMAT_IEEE_FLOAT;
    wfx.SubFormat = {0x00000003,0x0000,0x0010,0x80,0x00,0x00,0xAA,0x00,0x38,0x9B,0x71}; //KSDATAFORMAT_SUBTYPE_IEEE_FLOAT
    wfx.Format.wBitsPerSample = 32;
    wfx.Samples.wValidBitsPerSample = wfx.Format.wBitsPerSample;
    wfx.Format.nBlockAlign = wfx.Format.nChannels * wfx.Format.wBitsPerSample / 8;
    wfx.Format.nAvgBytesPerSec = wfx.Format.nBlockAlign * wfx.Format.nSamplesPerSec;
    wave_out_result = waveOutOpen( &WaveOutHandle, WAVE_MAPPER, &(wfx.Format), (DWORD_PTR) &WaveOutCallback, 0, CALLBACK_FUNCTION );
}
if( wave_out_result != MMSYSERR_NOERROR )
{
    if( userdata.HighRes )
        fprintf( stderr, "waveOutOpen returned %i%s when attempting float output\n", wave_out_result, (wave_out_result == WAVERR_BADFORMAT)?" (WAVERR_BADFORMAT)":"" );
    wfx.Format.wFormatTag = WAVE_FORMAT_PCM;
    wfx.SubFormat = {0x00000001,0x0000,0x0010,0x80,0x00,0x00,0xAA,0x00,0x38,0x9B,0x71}; //KSDATAFORMAT_SUBTYPE_PCM
    wfx.Format.wBitsPerSample = 16;
    wfx.Samples.wValidBitsPerSample = wfx.Format.wBitsPerSample;
    wfx.Format.nBlockAlign = wfx.Format.nChannels * wfx.Format.wBitsPerSample / 8;
    wfx.Format.nAvgBytesPerSec = wfx.Format.nBlockAlign * wfx.Format.nSamplesPerSec;
    wave_out_result = waveOutOpen( &WaveOutHandle, WAVE_MAPPER, &(wfx.Format), (DWORD_PTR) &WaveOutCallback, 0, CALLBACK_FUNCTION );
    if( wave_out_result == MMSYSERR_NOERROR )
        userdata.HighRes = false;
    else
        fprintf( stderr, "waveOutOpen returned %i%s when attempting int16 output\n", wave_out_result, (wave_out_result == WAVERR_BADFORMAT)?" (WAVERR_BADFORMAT)":"" );
}

If I set Format.nChannels = 2; dwChannelMask = 0x3; for stereo, the first waveOutOpen attempt using IEEE-float format fails with return code WAVERR_BADFORMAT, but the second try using PCM format succeeds.

If I try Format.nChannels = 4; dwChannelMask = 0x33; for quadraphonic, both IEEE-float and PCM waveOutOpen attempts fail with WAVERR_BADFORMAT.

However if I set Format.cbSize = 0; then everything works correctly with 2 channels in either format, which makes sense since that is essentially what I'd been doing with WAVEFORMATEX before. But this does not work with 4 channels.

What have I gotten wrong here? My ultimate goal is quadraphonic or 5.1 surround output in IEEE-float format. I'm especially baffled why I can't even get stereo IEEE-float output to work using WAVEFORMATEXTENSIBLE but it works perfectly with WAVEFORMATEX.

Answer 1

If are using a WAVEFORMATEXTENSIBLE structure, you have to indicate that by setting the correct format tag in the "base" WAVEFORMATEX.

wfx.Format.wFormatTag = WAVE_FORMAT_EXTENSIBLE;

See the documentation of the Format member of WAVEFORMATEXTENSIBLE.

This makes it possible for the called code (that only has a pointer to a WAVEFORMATEX) to discern if it is dealing with a WAVEFORMATEX or an WAVEFORMATEXTENSIBLE structure.
In the case of WAVEFORMATEXTENSIBLE the actual audio format is then uniquely identified by the SubFormat member.