zero latency microphone loopback, sidetone

Question

I am attempting to programatically create a zero latency microphone loopback to speakers or output. This is used to generate sidetone for a headset. As I am sure any reader is aware, sidetone must be zero latency otherwise, hearing yourself with a delay causes you to lose most of your coherent speaking abilities.

I've tried to create solutions using both Naudio in C# and portaudio with C++. I have had the best luck with PortAudio, however I cannot achieve the zero latency sidetone that I need. Portaudio yields a 5ms or so delay which is detectable and causes my speech to continuously slow down.

I do know that windows provides a microphone loopback and I have tested this however even the windows loopback has enough delay to be annoying as sidetone.

I ask my question in two parts

1.) Is this a limitation of the hardware/software and is achieving zero audio latency practically insurmountable?

2.) Here is my C++ code using portaudio. Is there any way to decrease the latency more than I already have? (please pardon my possibly sloppy code, I'm pretty new to C++ and I am still learning)

class vC_sidetone {
public:
    void enable(int in_ch, int out_ch);
    void disable();

    vC_sidetone(int inputDevice, int outputDevice){
        st_inputDevice = inputDevice;
        st_outputDevice = outputDevice;
    }

    int st_instanceCallBack(const void *inputBuffer,
        void *outputBuffer,
        unsigned long framesPerBuffer,
        const PaStreamCallbackTimeInfo *timeInfo,
        PaStreamCallbackFlags statusFlags);

private:
    int st_inputDevice;
    int st_outputDevice;
    PaError st_error;
    PaStream *st_stream = NULL;
};

PaStreamParameters vC_getParam(PaDeviceIndex dev, int ch, PaSampleFormat smplFormat){

    PaStreamParameters param;
    param.device = dev;
    param.channelCount = ch;
    param.sampleFormat = smplFormat;
    //param.suggestedLatency = Pa_GetDeviceInfo(dev)->defaultLowInputLatency;
    param.suggestedLatency = 0.000; //here is a good place to tweak latency
    param.hostApiSpecificStreamInfo = NULL;

    return param;
}

void vC_sidetone::enable(int in_ch, int out_ch){
    int framesPerBuffer = 1;
    PaSampleFormat smplFormat = paFloat32;
    int smplRate = 44100;

    PaStreamParameters inParam = vC_getParam(st_inputDevice, in_ch, smplFormat);
    PaStreamParameters outParam = vC_getParam(st_outputDevice, out_ch, smplFormat);

    st_error = Pa_Initialize();

    // Open and start stream using callback:
    st_error = Pa_OpenStream(
        &st_stream,
        &inParam,
        &outParam,
        smplRate,
        framesPerBuffer,
        paClipOff,
        st_gblCallBack,
        this
        );

    st_error = Pa_StartStream(st_stream);
}

void vC_sidetone::disable(){

    st_error = Pa_StopStream(st_stream);
    Pa_AbortStream(st_stream);
    Pa_CloseStream(st_stream);
    Pa_Terminate();
}

int vC_sidetone::st_instanceCallBack(const void *inputBuffer,
                            void *outputBuffer,
                            unsigned long framesPerBuffer,
                            const PaStreamCallbackTimeInfo *timeInfo,
                            PaStreamCallbackFlags statusFlags){

    (void)timeInfo; // Prevent unused variable warnings.
    (void)statusFlags;


    // Cast data to floats:
    float *out = (float*)outputBuffer;
    float *in = (float*)inputBuffer;
    unsigned long i;

    //for (i = 0; i < framesPerBuffer*NUM_CHANNELS; i++)
    //  out[i] = in[i];
    for (i = 0; i < framesPerBuffer; i++)  //another good place for latency
        out[i] = in[i];

    return paContinue;
}

//this is actually not part of the vC_sidetone class, I was getting linker errors
static int st_gblCallBack(const void *inputBuffer,
    void *outputBuffer,
    unsigned long framesPerBuffer,
    const PaStreamCallbackTimeInfo *timeInfo,
    PaStreamCallbackFlags statusFlags,
    void *userData){

    return ((vC_sidetone*)userData)->st_instanceCallBack(inputBuffer, outputBuffer,
        framesPerBuffer,
        timeInfo,
        statusFlags);
}

Answer 1

Abstraction layers can only add latency, not reduce it. Pretty obvious, time travel is impossible. PulseAudio is not the native API on Windows, it's built on top of the native audio architecture.

The current native Windows API is WASAPI. It's got a pretty good reputation for being low-latency. But it too is an abstraction layer over the underlying hardware. USB for instance also has latency. You can't do much better than that.