Program interruptions in Keyboardhook program lock up Windows

Question

I am writing a program and using SetWindowsHookEx to set a global Keyboard hook. Whenever I interrupt my program (for example, using a breakpoint or an exception gets thrown), windows completely locks up in a sense that no inputs are handled anymore, except for ctrl+alt+del. I cannot even use the task manager to close down the application.

Obviously, this is rather bad for developing a complex application and I would like to know what workarounds exist here? Is there some way to prevent the hooks to lock up windows like that? I am using a DLL written in C++ in C# via PInvoke. Setting a Hook is done via

EXPORT BOOL CALLBACK SetupHook (HWND hParent){
   hWindow = hParent;
   hhkHook = SetWindowsHookEx(WH_KEYBOARD, KeyboardHookProc, hDllInstance, NULL);
   hhkMouseHook = SetWindowsHookEx(WH_MOUSE, MouseHookProc, hDllInstance, NULL);
   return TRUE;
}

And then the message is redirected with

LRESULT CALLBACK KeyboardHookProc (int nCode, WPARAM wParam, LPARAM lParam){
    if (nCode == HC_ACTION){
        if ((lParam & 1073741824) != 1073741824){
           SendMessage ((HWND) hWindow, (WM_USER + 2), wParam, lParam);
        }
    }
    return CallNextHookEx (hhkHook, nCode, wParam, lParam);
}

Answer 1

These types of hooks usually inject a function into the hook chain.
The hook chain is managed by each link in the chain.

What is happening here is your function is blocked because of your breakpoint which means that your function (the dll you are pinvoking) can not call the next keyboard hook in the chain. So it appears that your inputs have become frozen.

As you are also hooking the mouse - the same thing happens there and you now have no way to provide input to the IDE to tell it to keep running.

Ctl+Alt+Del is a secure key sequence handled by the kernel, so there's pretty much nothing you can do to prevent that sequence from being interrupted.

I would suggest you debug your mouse and keyboard hooks separately, keeping in mind what you have learned here. (e.g. if you terminate your program before restoring the original hook chain you could lose your keyboard/mouse until the next logoff/logon).