How to Properly Set a QCursor with a Mask？

Question

I am implementing a remote desktop protocol and need to display the cursor bitmap from the remote desktop on the local client. This involves setting mouse pointer types with monochrome bitmaps. Additionally, I require cross-platform functionality, which is why I am using QCursor in Qt.

On windows platform. I have now learned how to set a cursor with a mask using the Win32 API.And I have achieved the desired result.

Due to the lengthy binary content, I haven't posted the code here. For reference, you can see this example: https://learn.microsoft.com/en-us/windows/win32/menurc/using-cursors

HINSTANCE inst;
HCURSOR cursor = CreateCursor(inst,
                              pCursorData->xHotspot,
                              pCursorData->yHotspot,
                              pCursorData->width,
                              pCursorData->height,
                              (void*)pCursorData->pixelData,
                              (void*)(pCursorData->pixelData + size));

SetCursor(cursor);

Afterward, I utilized QImage to convert my monochrome bitmap and proceeded to configure QCursor. However, the result I obtained differed from what is typically achieved when employing the native WIN32 API interface.

// `height` is 2 bitmap;
int size =  pCursorData->width * pCursorData->height / 2 / 8;
QImage     image(pCursorData->pixelData,          pCursorData->width,pCursorData->height / 2,QImage::Format_Mono);
QImage maskImage((pCursorData->pixelData + size), pCursorData->width,pCursorData->height / 2,QImage::Format_Mono);

pixmap = QPixmap::fromImage(image);
pixmap.setMask(maskBitmap);
QCursor cursor(pixmap, pCursorData->xHotspot, pCursorData->yHotspot);

renderWidget->setCursor(cursor);

Ultimately, after consulting additional documentation, I revised the code accordingly.

/**
 * type: MONOCHAROME has (AND bitmap) and (XOR bitmap).
 *  1 bit = 1 pixed
 *
 * Windows:
 * AND bitmap  XOR bitmap  Display
 *   0          0           Black
 *   0          1           White
 *   1          0           Screen
 *   1          1           Reverse screen
 *
 * QCursor:
 * The cursor bitmap (B) and mask (M) bits are combined like this:
 *  B=1 and M=1 gives black.
 *  B=0 and M=1 gives white.
 *  B=0 and M=0 gives transparent.
 *  B=1 and M=0 gives an XOR'd result under Windows, undefined results on all other platforms.
 * Use the global Qt color Qt::color0 to draw 0-pixels and Qt::color1 to draw 1-pixels in the bitmaps.
 */

int pitch = pCursorData->dataLen / pCursorData->height;
bool andPixel;
bool xorPixel;
// `height` is 2 bitmap;
for(int y=0; y < pCursorData->height / 2; y++)
{
    for(int x=0; x < pCursorData->width; x++)
    {
        quint8 bitMask = 0x80 >> (x % 8);
        quint8 uint8_x = x / 8; // width is bit size

        andPixel = pCursorData->pixelData[y * pitch + uint8_x] & bitMask;
        xorPixel = pCursorData->pixelData[(y + pCursorData->height / 2) * pitch + uint8_x] & bitMask;

        if (!andPixel && !xorPixel) {
            image.setPixelColor(x, y, QColor(Qt::color1));
            maskImage.setPixelColor(x, y, QColor(Qt::color1));
        } else if (!andPixel && xorPixel) {
            image.setPixelColor(x, y, QColor(Qt::color0));
            maskImage.setPixelColor(x, y, QColor(Qt::color1));
        } else if (andPixel && !xorPixel) {
            image.setPixelColor(x, y, QColor(Qt::color0));
            maskImage.setPixelColor(x, y, QColor(Qt::color0));
        } else if (andPixel && xorPixel) {
            image.setPixelColor(x, y, QColor(Qt::color1));
            maskImage.setPixelColor(x, y, QColor(Qt::color0));
        }
    }
}

pixmap = QPixmap::fromImage(image);
pixmap.setMask(maskBitmap);
QCursor cursor(pixmap, pCursorData->xHotspot, pCursorData->yHotspot);

renderWidget->setCursor(cursor);

Despite all my efforts, they proved to be in vain. I later delved into the Qt source code to understand the implementation of QCursor and discovered it involved numerous bitwise operations, which were quite challenging for me to grasp in a short period. Therefore, I am reaching out to the experts in this field for guidance: How should one convert Windows' AND mask and XOR mask bitmaps into a QCursor?

Answer 1

I have made continuous attempts and discovered some patterns in color conversion. The official documentation of QCursor mentions Qt::color0 and Qt::color1, which indeed follow certain rules. However, I found that the results I obtained are the opposite of the bit values.

As mentioned in my comments, I am currently inverting the bit values of the Windows original AND mask and XOR mask to match Qt::color0 and Qt::color1 as mentioned in the QCursor documentation.

        /**
         * type: MONOCHAROME has (AND bitmap) and (XOR bitmap).
         *  1 bit = 1 pixed
         *
         * Windows:
         * AND bitmap  XOR bitmap  Display
         *   0          0           Black
         *   0          1           White
         *   1          0           Screen
         *   1          1           Reverse screen
         *
         * QCursor:
         * The cursor bitmap (B) and mask (M) bits are combined like this:
         *  B=1 and M=1 gives black.
         *  B=0 and M=1 gives white.
         *  B=0 and M=0 gives transparent.
         *  B=1 and M=0 gives an XOR'd result under Windows, undefined results on all other platforms.
         * Use the global Qt color Qt::color0 to draw 0-pixels and Qt::color1 to draw 1-pixels in the bitmaps.
         *
         * Actual results:
         *   0          0           Black
         *   1          0           White
         *   1          1           transparent screen
         *   0          1           Reverse screen
         */

        int size =  pCursorData->width * pCursorData->height / 2 / 8;
        if(pCursorData->dataLen / 2 != size)
        {
            qWarning() << "[CMouseModule::SetServerCursor] set Cursor Data dataLen failed.";
            return;
        }

        QImage     image(pCursorData->width, pCursorData->height / 2, QImage::Format_Mono);
        QImage maskImage(pCursorData->width, pCursorData->height / 2, QImage::Format_Mono);

        if(image.sizeInBytes() != size)
        {
            qWarning() << "[CMouseModule::SetServerCursor] bitmap size not eq Cursor Data image size.";
            return;
        }

        uchar* imageBits = image.bits();
        uchar* maskImageBits = maskImage.bits();

        int pitch = pCursorData->dataLen / pCursorData->height;
        bool andPixel;
        bool xorPixel;
//        // `height` is 2 bitmap;
        for(int y=0; y < pCursorData->height / 2; y++)
        {
            for(int x=0; x < pCursorData->width; x++)
            {
                quint8 bitMask = 0x80 >> (x % 8);
                quint8 uint8_x = x / 8; // width is bit size

                andPixel = pCursorData->pixelData[y * pitch + uint8_x] & bitMask;
                xorPixel = pCursorData->pixelData[(y + pCursorData->height / 2) * pitch + uint8_x] & bitMask;

                if(!andPixel && !xorPixel)
                {
                    imageBits[y * pitch + uint8_x] &= ~bitMask; // 0
                    maskImageBits[y * pitch + uint8_x] &= ~bitMask; // 0
                } else if(!andPixel && xorPixel)
                {
                    imageBits[y * pitch + uint8_x] |= bitMask; // 1
                    maskImageBits[y * pitch + uint8_x] &= ~bitMask; // 0
                } else if(andPixel && !xorPixel)
                {
                    imageBits[y * pitch + uint8_x] |= bitMask; // 1
                    maskImageBits[y * pitch + uint8_x] |= bitMask; // 1
                } else if(andPixel && xorPixel)
                {
                    imageBits[y * pitch + uint8_x] &= ~bitMask; // 0
                    maskImageBits[y * pitch + uint8_x] |= bitMask; // 1
                }
            }
        }

        QBitmap bitmap;
        QBitmap maskBitmap;
        bitmap = QBitmap::fromImage(image);
        maskBitmap = QBitmap::fromImage(maskImage);
        QCursor cursor(bitmap, maskBitmap, pCursorData->xHotspot, pCursorData->yHotspot);
renderWidget->setCursor(cursor);

I'm not sure why it happened, but it's indeed functioning properly now. If you know the reason behind it, please continue to reply.