PInvoke PostMessage not working across user accounts

Question

I'm building a kill switch into my application so that only one instance can run at a time on a single computer. I'm accomplishing this by posting messages between the process of the running application and the process of the new instance of the application:

[DllImport("user32.dll", EntryPoint = "PostMessageA", CharSet = CharSet.Ansi, SetLastError = true, ExactSpelling = true)]
private static extern int PostMessage(int hwnd, int wMsg, int wParam, int lParam);

I can't use Process.Kill because if another user is running the application and the current user does not have sufficient privileges I get issues.

When running 2 instances under the same user account I have no issues. Messages are sent correctly and received correctly. However, when running one instance from one user account, then switching users and running a second instance, my first instance is not receiving the messages.

Here's my logic for subscribing to window messages:

var wih = new WindowInteropHelper(this);
var hwndSource = HwndSource.FromHwnd(wih.Handle);
var hwndSourceHook = new HwndSourceHook(HookHandler);

if (hwndSource != null)
    hwndSource.AddHook(hwndSourceHook);

And here's my hook handler:

private IntPtr HookHandler(IntPtr hwnd, int msg, IntPtr wParam, IntPtr lParam, ref bool handled)
    {
        handled = false;

        switch (msg)
        {
            case 0x400: // interprocess message received
                App.InterprocessManager.BuildString(lParam);
                break;
        }
        return IntPtr.Zero;
    }

Here's the logic to send the message:

private void SendString()
    {
        //create byte array
        byte[] ba = null;
        //encode string to byte array
        if (object.ReferenceEquals(enc, Encoding.UTF8))
        {
            ba = Encoding.UTF8.GetBytes(lParam);
        }
        else if (object.ReferenceEquals(enc, Encoding.Unicode))
        {
            ba = Encoding.Unicode.GetBytes(lParam);
        }
        else if (object.ReferenceEquals(enc, Encoding.ASCII))
        {
            ba = Encoding.ASCII.GetBytes(lParam);
        }
        else
        {
            ba = Encoding.Default.GetBytes(lParam);
        }
        int i = 0;
        for (i = 0; i <= ba.Length - 1; i++)
        {
            //start post message
            PostMessage(hwnd, wMsg, wParam, ba[i]);
        }
        //post a terminator message to destination window
        PostMessage(hwnd, wMsg, wParam, 0);
    }

No Win32 errors are being set by the PostMessage function. I can't seem to find any documentation regarding posting messages between processes across user accounts. Is this something that can't actually be done?

Answer 1

You cannot send window messages to a process in a different session. You must use an IPC mechanism that is based on kernel rather than user objects.

In your scenario, one named mutex and one named semaphore is probably all you need. Note that the names must have a Global\ prefix in order to allow multiple sessions to share a single object.

On process startup, first create or open the semaphore. Set the initial count to zero.

Then create the named mutex with bInitialOwner set to TRUE and check the last error code to see if the mutex already existed.

If the mutex did not already exist, you are the first instance, so create a thread to wait on the semaphore. If the semaphore is signaled, exit the process. Make sure you don't release the mutex until it is safe for the other process to run; if in doubt, don't release it at all. Windows will signal it as abandoned once the process has exited.

If the mutex did already exist, another instance is running. Signal the semaphore, then wait on the mutex. Once you've obtained ownership of the mutex it is safe to continue. You then need to create a thread to wait on the semaphore, in case yet another instance comes along and wants to bump you.

If you need a more complicated communications channel between the processes, for example if you need to hand over the current state of the communications channel, then your best choice is probably named pipes.