GetQueuedCompletionStatus returns with bad information

774 views Asked by At

I'm writing a multicast producer and multicast consumer applications using IOCP. My producer app is multi-threaded and publishes to multiple multicast channels. Each channel is published on its own thread, so I'm certain there are no synchronization / race condition issues with the send buffers. In fact, using wire shark, the datagrams look good (with an occasional one being malformed (unsure if this is normal for UDP datagrams)).

My consumer application is where I'm seeing the trouble. My consumer app is also multithreaded. But again, only a single WSARecv is called for any single multicast channel (each channel has its own set of buffers), so there is only a single thread accessing the buffers. When my call to GetQueuedCompletionStatus returns, I process the data, then call WSARecv to start the process again.

The problem I see is that GetQueuedCompletionStatus returns with old data in the buffer (I reuse the buffer). When I write the buffers to a log file, I can see that the data is not changing, though my Wire Shark recordings show the datagrams to be fine. The intermittent nature of this bug seems to point to a synchronization issue, but I've looked carefully through the library AND I don't see this issue using TCP. Is there any way that GetQueuedCompletionStatus could return bogus data?

bool cont = true;
DWORD xferd;
OVERLAPPED* overlapped = NULL;
TTOverlapped<InternalEndpointPtr>* tto;
ULONG_PTR key;

while (cont) {
    BOOL b = GetQueuedCompletionStatus(iocp_, &xferd, &key, &overlapped, INFINITE);

    // only a single WSARecv called per overlapped structure
    // it's assumed that if overlapped != NULL, WSARecv call has completed
    if (overlapped) {
        tto = static_cast<TTOverlapped<InternalEndpointPtr>*>(overlapped);

        switch (tto->opCode()) {
        case IOCP_OP_CODE_READ: readMessage(tto, xferd); break;
        case IOCP_OP_CODE_WRITE:
            if (xferd == 0)
                std::printf("Send operation failed: %d.\n", WSAGetLastError());
            break;
        case IOCP_OP_CODE_NEW_CLIENT:
            tto->endpoint()->recv();
            delete tto;
            break;
        case IOCP_OP_CODE_SHUTDOWN:
            cont = false;
            break;
        default
            std::printf("Not expecting this: %d!!!\n", tto->opCode());
            break;
        }
    }
    else {
        DWORD d = GetLastError();
        if (d != 995)
            Logger::log(LOG_LEVEL_STATUS, LOG_TYPE_SYSTEM, "InternalIpcProcessor::processIocpCallbacks(): GetQueuedCompletionStatus returned FALSE. Thread id = %d. GetLastError = %d.", ::GetCurrentThreadId(), GetLastError());
    }
}
0

There are 0 answers