Converting AvAudioInputNode to S16LE PCM

Question

I'm trying to convert input node format to S16LE format. I've tried it with AVAudioMixerNode

First I create audio session

do {
  try audioSession.setCategory(.record)
  try audioSession.setActive(true)
} catch {
  ...
}

//Define formats
let inputNodeOutputFormat = audioEngine.inputNode.outputFormat(forBus: 0)
guard let wantedFormat = AVAudioFormat(commonFormat: AVAudioCommonFormat.pcmFormatInt16, sampleRate: 16000, channels: 1, interleaved: false) else {
  return;
}

//Create mixer node and attach it to the engine
audioEngine.attach(mixerNode)

//Connect the input node to mixer node and mixer node to mainMixerNode
audioEngine.connect(audioEngine.inputNode, to: mixerNode, format: inputNodeOutputFormat)
audioEngine.connect(mixerNode, to: audioEngine.mainMixerNode, format: wantedFormat)

//Install the tab on the output of the mixerNode
mixerNode.installTap(onBus: 0, bufferSize: bufferSize, format: wantedFormat) { (buffer, time) in
  let theLength = Int(buffer.frameLength)
  var bufferData: [Int16] = []
  for i in 0 ..< theLength
  {
    let char = Int16((buffer.int16ChannelData?.pointee[i])!)
    bufferData.append(char)
  }
}

I get the following error.

Exception 'I[busArray objectAtindexedSubscript:
(NSUlnteger)element] setFormat:format
error:&nsErr]: returned false, error Error
Domain=NSOSStatusErrorDomain Code=-10868
"(null)"' was thrown

What part of the graph did I mess up?

Answer 1

You have to set the format of nodes to match the actual format of the data. Setting the node's format doesn't cause any conversions to happen, except that mixer nodes can convert sample rates (but not data formats). You'll need to use an AVAudioConverter in your tap to do the conversion.

As an example of what this code would look like, to handle arbitrary conversions:

let inputNode = audioEngine.inputNode
let inputFormat = inputNode.inputFormat(forBus: 0)
let outputFormat = AVAudioFormat(... define your format ...)

guard let converter = AVAudioConverter(from: inputFormat, to: outputFormat) else {
    throw ...some error...
}

inputNode.installTap(onBus: 0, bufferSize: 1024, format: inputFormat) {[weak self] (buffer, time) in
    
    let inputBlock: AVAudioConverterInputBlock = {inNumPackets, outStatus in
        outStatus.pointee = AVAudioConverterInputStatus.haveData
        return buffer
    }
    
    let targetFrameCapacity = AVAudioFrameCount(outputFormat.sampleRate) * buffer.frameLength / AVAudioFrameCount(buffer.format.sampleRate)
    if let convertedBuffer = AVAudioPCMBuffer(pcmFormat: outputFormat, frameCapacity: targetFrameCapacity) {
        var error: NSError?
        let status = converter.convert(to: convertedBuffer, error: &error, withInputFrom: inputBlock)
        assert(status != .error)
        
        let sampleCount = convertedBuffer.frameLength
        let rawData = convertedBuffer.int16ChannelData![0]

        // ... and here you have your data ...
    }
}

If you don't need to change the sample rate, and you're converting from uncompressed audio to uncompressed audio, you may be able to use the simpler convert(to:from:) method in your tap.

Since iOS 13, you can also do this with AVAudioSinkNode rather than a tap, which can be more convenient.