Acoustic Echo Cancellation (AEC) on Xamarin.iOS

Question

I am currently working on cross platform Voip app development based on Xamarin.iOS platform. I did search about the AEC implementation on iOS but most topics are related to Objective-C. What I have already achieved now is: I could use Audiotoolbox (audio queue) to obtain the input raw sound data from mic and send it out with socket. But during testing with other device, I am experiencing very clear echo on the phone. Here is code:

private void SetupInputQueue()
    {
        inputQueue = new InputAudioQueue(audioStreamBasicDescription);
        //sendbuffer initialization
        heading = Encoding.ASCII.GetBytes("msg ");
        sendBuffer = new byte[CountAudioBuffers][];
        for (int i = 0; i < CountAudioBuffers; i++)
        {
            sendBuffer[i] = new byte[516];
            for (int j = 0; j < heading.Length; j++)
            {
                sendBuffer[i][j] = heading[j];
            }
        }

        for (int count = 0; count < CountAudioBuffers; count++)
        {
            IntPtr bufferpointer;
            inputQueue.AllocateBuffer(AudioBufferLength, out bufferpointer);
            inputQueue.EnqueueBuffer(bufferpointer, AudioBufferLength, null);
        }
        inputQueue.InputCompleted += HandleInputCompleted;
    }

    private void HandleInputCompleted(object sender, InputCompletedEventArgs e)
    {
        unsafe
        {
            byte* shortPtr = (byte*)e.UnsafeBuffer->AudioData;
            for (int count = heading.Length; count < sendBuffer[sendOutIndex].Length; count++)
            {
                sendBuffer[sendOutIndex][count] = *shortPtr;
                shortPtr++;
            }
        }
        socket.SendTo(sendBuffer[sendOutIndex], master);
        this.inputQueue.EnqueueBuffer(e.IntPtrBuffer, AudioBufferLength, null);
        sendOutIndex = (sendOutIndex + 1) % CountAudioBuffers;
    }

According to AEC on OSX using AudioQueue, I learnt the hint that I should pass the raw sound data to I/O unit (audio unit?). But due to the lack of examples in Xamarin.iOS(c#), I can't figure out how to implement this process in detail. Could any one familiar with Voip app development on Xamarin platform give me some example to study? Very appreciate any help or hint on this.

(Nov, 21, 2018)Some related posts I found:Record audio with audio unit Audio unit Callbacks An audio unit example

Answer 1

Finally we figured out how to turn on the iOS build-in AEC on Xamarin.iOS platform. The documentation here helped a lot to have a better understanding of the details of Audio Unit (even though this document is for iOS native development environment) and the working mechanism. If you want to precisely control every audio unit and understand parameters in the functions, this document is a must read.

The work here provide a very nice start point to do experiment on Audio Unit. I made some modifications in step 2 of this post. The key part is that we have to use AudioTypeOutput.VoiceProcessingIO, this turns on AEC.

public void prepareAudioUnit()
    {

        var _audioComponent =   AudioComponent.FindComponent(AudioTypeOutput.VoiceProcessingIO);

        audioUnit = new AudioUnit.AudioUnit(_audioComponent);

        audioUnit.SetEnableIO(true,
            AudioUnitScopeType.Input,
            1 // Remote Input
        );

        // setting audio format
        audioUnit.SetAudioFormat(audioStreamBasicDesc,
            AudioUnitScopeType.Output,
            1
        );                                    

        audioUnit.SetInputCallback(input_CallBack, AudioUnitScopeType.Input, 1);

        audioUnit.SetRenderCallback(render_CallBack, AudioUnitScopeType.Global, 0);


        audioUnit.Initialize();
        audioUnit.Start();
    }

Answer 2

I suggest to analyze the nature of the echo you are having in your system (for example test the echo path). Perhaps it is outside the capabilities of your built-in echo cancellation.