Weird P/Invoke issue on Win 7 x64

Question

I'm P/Invoking to CreateRectRgn in gdi32.dll. The normal P/Invoke signature for this function is:

[DllImport("gdi32", SetLastError=true)]
static extern IntPtr CreateRectRgn(int nLeft, int nTop, int nRight, int nBottom);

As a shortcut, I've also defined this overload:

[DllImport("gdi32", SetLastError=true)]
static extern IntPtr CreateRectRgn(RECT rc);

[StructLayout(LayoutKind.Sequential)]
struct RECT{
    public int left;
    public int top;
    public int right;
    public int bottom;
}

(Yes, I am aware of CreateRectRgnIndirect, but since I must use functions to convert between System.Drawing.Rectangle and this RECT structure, the above is more useful to me, as it doesn't involve an intermediate variable.)

This overload should work identically to the normal signature, since it should put the stack in an identical state at entry to CreateRectRgn. And indeed, on Windows XP, 32-bit, it works flawlessly. But on Windows 7, 64-bit, the function returns zero, and Marshal.GetLastWin32Error() returns 87, which is "The parameter is incorrect."

Any ideas as to what could be the problem?

Answer 1

Oh. The calling convention Microsoft uses on x64 is totally different from STDCALL. In the call to CreateRectRgn, the stack isn't used for the parameters at all, they're all passed in registers. When I try to pass a RECT structure, it makes a copy of the structure on the stack, and puts a pointer to this copy in a register. Therefore, this little trick won't work at all in 64-bit Windows. Now I've got to go through all my interop code and find other places I've done this and take them all out.