The sound muted after playing audio with Audio Queue on iOS for a while

Question

I am coding a real time audio playback program on iOS.

It receives audio RTP packages from the peer, and put it into audio queue to play.

When start playing, the sound is OK. But after 1 or 2 minutes, the sound muted, and there is no error reported from AudioQueue API. The callback function continues being called normally, nothing abnormal.

But it just muted.

My callback function:

1: Loop until there is enough data can be copied to audio queue buffer

do 
{
    read_bytes_enabled = g_audio_playback_buf.GetReadByteLen();
    if (read_bytes_enabled >= kAudioQueueBufferLength)
    {
        break;
    }
    usleep(10*1000);
}
while (true);

2: Copy to AudioQueue Buffer, and enqueue it. This callback function keeps running normally and no error.

//copy to audio queue buffer
read_bytes = kAudioQueueBufferLength;

g_audio_playback_buf.Read((unsigned char *)inBuffer->mAudioData, read_bytes);

WriteLog(LOG_PHONE_DEBUG, "AudioQueueBuffer(Play): copy [%d] bytes to AudioQueue buffer! Total len = %d", read_bytes, read_bytes_enabled);

inBuffer->mAudioDataByteSize = read_bytes;

UInt32 nPackets = read_bytes / g_audio_periodsize; // mono

inBuffer->mPacketDescriptionCount = nPackets;

// re-enqueue this buffer
AudioQueueEnqueueBuffer(inAQ, inBuffer, 0, NULL);

Answer 1

The problem has been resolved.

The key point is, you can not let the audio queue buffer waits, you must keep feeding it, or it might be muted. If you don't have enough data, fill it with blank data.

so the following code should be changed:

do 
{
    read_bytes_enabled = g_audio_playback_buf.GetReadByteLen();
    if (read_bytes_enabled >= kAudioQueueBufferLength)
{
    break;
}
    usleep(10*1000);
}
while (true);

changed to this:

read_bytes_enabled = g_audio_playback_buf.GetReadByteLen();
if (read_bytes_enabled < kAudioQueueBufferLength)
{
    memset(inBuffer->mAudioData, 0x00, kAudioQueueBufferLength);
}
else
{
    inBuffer->mAudioDataByteSize = kAudioQueueBufferLength;
}
...

Answer 2

You can let the AudioQueue wait if you use AudioQueuePause.

In this exemple, in Swift 5, I use a generic queue. When this queue is empty, as you did, I fill my buffer with empty data in callback and call AudioQueuePause. It's important to note that all of AudioQueueBuffer send to AudioQueueRef with AudioQueueEnqueueBuffer before call AudioQueuePause are played.

Create an userData class to send everything you need to your callback :

class UserData {
    let dataQueue = Queue<Data>()
    let semaphore = DispatchSemaphore(value: 1)
}
private var inQueue: AudioQueueRef!
private var userData = UserData()

Give an instance of this class when you create your AudioQueue and start it :

AudioQueueNewOutput(&inFormat, audioQueueOutputCallback, &userData, nil, nil, 0, &inQueue)
AudioQueueStart(inQueue, nil)

Generate all your buffers and don't enqueue them directly : call your callback function :

for _ in 0...2 {
    var bufferRef: AudioQueueBufferRef!
    AudioQueueAllocateBuffer(inQueue, 320, &bufferRef)
    audioQueueOutputCallback(&userData, inQueue, bufferRef)
}

When you receive audio data, you can call a method who enqueue your data and let it wait for callback function get it :

func audioReceived(_ audio: Data) {
    let dataQueue = userData.dataQueue
    let semaphore = userData.semaphore

    semaphore.wait()
    dataQueue.enqueue(audio)
    semaphore.signal()
    // Start AudioQueue every time, if it's already started this call do nothing
    AudioQueueStart(inQueue, nil)
}

Finally you can implement a callback function like this :

private let audioQueueOutputCallback: AudioQueueOutputCallback = { (inUserData, inAQ, inBuffer) in
    // Get data from UnsageMutableRawPointer
    let userData: UserData = (inUserData!.bindMemory(to: UserData.self, capacity: 1).pointee)
    let queue = userData.dataQueue
    let semaphore = userData.semaphore
    // bind UnsafeMutableRawPointer to UnsafeMutablePointer<UInt8> for data copy
    let audioBuffer = inBuffer.pointee.mAudioData.bindMemory(to: UInt8.self, capacity: 320)

    if queue.isEmpty {
        print("Queue is empty: pause")
        AudioQueuePause(inAQ)
        audioBuffer.assign(repeating: 0, count: 320)
        inBuffer.pointee.mAudioDataByteSize = 320
    } else {
        semaphore.wait()
        if let data = queue.dequeue() {
            data.copyBytes(to: audioBuffer, count: data.count)
            inBuffer.pointee.mAudioDataByteSize = data.count
        } else {
            print("Error: queue is empty")
            semaphore.signal()
            return
        }
        semaphore.signal()
    }

    AudioQueueEnqueueBuffer(inAQ, inBuffer, 0, nil)
}

In my case I use 320 bytes buffer for 20ms of PCM data 16bits, 8kHz, mono. This solution is more complexe but better than a pseudo infinite loop with empty audio data for your CPU. Apple is very punitive with greedy apps ;)

I hope this solution will help.