Potential kind of asynchronous (overlapped) I/O implementation in Windows

Question

I would like to discuss potential kind of asynchronous (Overlapped) I/O implementations in Windows, because there are many ways to implement this. Overlapped I/O in Windows provides the ability to process data asynchronously, ie the execution of the operations are nonblocking.

Edit: The purpose of this question is the discussion about improvement of my own implementation on the one hand, and the discussion of alternate implementation on the other hand. What asynchronous I/O implementation would make most sense on parallel heavy I/O, what make most sense in small mostly single threaded application.

I will cite MSDN:

When a function is executed synchronously, it does not return until the operation has been completed. This means that the execution of the calling thread can be blocked for an indefinite period while it waits for a time-consuming operation to finish. Functions called for overlapped operation can return immediately, even though the operation has not been completed. This enables a time-consuming I/O operation to be executed in the background while the calling thread is free to perform other tasks. For example, a single thread can perform simultaneous I/O operations on different handles, or even simultaneous read and write operations on the same handle.

I assume that the reader is familiar with the basic concept of overlapped I/O.

Another solution for asynchronous I/O are completions ports, but this shall not be the subject of this discussion. More information on other I/O concepts can be found on MSDN "About File Management > Input and Output (I/O) > I/O Concepts"

I would like to present my (C/C++) implementation here and share it for discussion.

This is my extended OVERLAPPED struct called IoOperation:

struct IoOperation : OVERLAPPED {
    HANDLE Handle;
    unsigned int Operation;
    char* Buffer;
    unsigned int BufferSize;
}

This struct is created each time an asynchronous operation like ReadFile or WriteFile is called. The Handle field shall be initialized with the corresponding device/file handle. Operation is a user defined field that tells what operation was called. The field Buffer is a pointer to a previously allocated chunk of memory with the given size BufferSize. Of course, this struct can be expanded at will. It could contain the operation result, acutaully transfered size etc.

The first thing we need is an (auto reset) event handle to be signaled each time an overlapped I/O is completed.

HANDLE hEvent = CreateEvent(0, FALSE, FALSE, 0);

First I decided to use only one event for all asynchronous operations. Then I decided to register this event with a thread pool thread with RegisterWaitForSingleObject.

HANDLE hWait = 0;
....
RegisterWaitForSingleObject(
    &hWait,
    hEvent,
    WaitOrTimerCallback,
    this,
    INFINITE,
    WT_EXECUTEINPERSISTENTTHREAD | WT_EXECUTELONGFUNCTION
);

So each time this event is signaled, my callback WaitOrTimerCallback is called.

An asynchronous operation is initialized like this:

IoOperation* Io = new IoOperation(hFile, hEvent, IoOperation::Write, Data, DataSize);
if (IoQueue->Enqueue(Io)) {
    WriteFile(hFile, Io->Buffer, Io->BufferSize, 0, Io);
}

Each operation is queued and is removed after successful GetOverlappedResult call in my WaitOrTimerCallback callback. Instead calling new all the time here, we could use a memory pool to avoid memory fragmentation and to make allocation faster.

VOID CALLBACK WaitOrTimerCallback(PVOID Parameter, BOOLEAN TimerOrWaitFired) {
    list<IoOperation*>::iterator it = IoQueue.begin();
    while (it != IoQueue.end()) {
        bool IsComplete = true;
        DWORD Transfered = 0;

        IoOperation* Io = *it;
        if (GetOverlappedResult(Io->Handle, Io, &Transfered, FALSE)) {
            if (Io->Operation == IoOperation::Read) {
                // Handle Read, virtual OnRead(), SetEvent, etc.
            } else if (Io->Operation == IoOperation::Write) {
                // Handle Read, virtual OnWrite(), SetEvent, etc.
            } else {
                // ...
            }
        } else {
            if (GetLastError() == ERROR_IO_INCOMPLETE) {
                IsComplete = false;
            } else {
                // Handle Error
            }
        }
        if (IsComplete) {
            delete Io;
            it = IoQueue.erase(it);
        } else {
            it++;
        }
    }
}

Of course, to be multi threading safe, we need a lock protection (critical section) when accessing the I/O queue for example.

There are advantages but also disadvantage of this kind of implementation.

Advantages:

• Execution in persistent thread pool thread, no manual thread creation is required
• Only one event is required
• Each operation is queued in an I/O queue (CancelIoEx can be called later)

Disadvantages:

• I/O queue requires extra memory/cpu time
• GetOverlappedResult is called for all queued I/O's even incompleted ones