Using jna to consume key events

Question

In one of the applications I am writing i need to consume certain key events so other applications dont process them.

In my code i make a com.sun.jna.platform.win32.WinUser.LowLevelKeyboardProc like so:

import com.sun.jna.Native;
import com.sun.jna.platform.win32.Kernel32;
import com.sun.jna.platform.win32.WinDef.HMODULE;
import com.sun.jna.platform.win32.WinDef.LRESULT;
import com.sun.jna.platform.win32.WinDef.WPARAM;
import com.sun.jna.platform.win32.WinUser.HHOOK;
import com.sun.jna.platform.win32.WinUser.KBDLLHOOKSTRUCT;
import com.sun.jna.platform.win32.WinUser.LowLevelKeyboardProc;
import com.sun.jna.platform.win32.User32;
import com.sun.jna.platform.win32.WinUser;

public class KeyHook implements Runnable{

private static volatile boolean quit = false;
private static HHOOK hhk;
private static LowLevelKeyboardProc keyboardHook;

private Main main;
User32 lib;
HMODULE hMod;
public boolean isHooked = false;

public KeyHook(final Main main) {
    this.main = main;
    lib = User32.INSTANCE;
    hMod = Kernel32.INSTANCE.GetModuleHandle(null);
    Native.setProtected(true);
}

@Override
public void run() {
    keyboardHook = new LowLevelKeyboardProc() {
        public LRESULT callback(int nCode, WPARAM wParam, KBDLLHOOKSTRUCT info) {
            if (nCode >= 0 && main.getPane().getTabCount() > 0) {
                switch (wParam.intValue()) {
                    case WinUser.WM_KEYUP:
                        if(info.vkCode == main.getListenMouse()){
                            main.listen();
                            return new LRESULT(1);
                        }
                        else if(info.vkCode == main.getStopListenMouse()){
                            main.stopListening();
                            return new LRESULT(1);
                        }
                        else if(info.vkCode == main.getStart()){
                            main.start();
                            return new LRESULT(1);
                        }
                        else if(info.vkCode == main.getPause()){
                            main.pause();
                            return new LRESULT(1);
                        }
                        else if(info.vkCode == main.getStop()){
                            main.stopRunning();
                            return new LRESULT(1);
                        }
                        else if(info.vkCode == 0x7B){
                            main.nextTab();
                            return new LRESULT(1);
                        }
                        break;
                    case WinUser.WM_KEYDOWN:
                       break;
                    case WinUser.WM_SYSKEYUP:
                        break;
                    case WinUser.WM_SYSKEYDOWN:
                        quit = true;
                        break;
                }
            }
            return lib.CallNextHookEx(hhk, nCode, wParam, info.getPointer());
            //return new LRESULT(1);
        }
    };
    hhk = lib.SetWindowsHookEx(WinUser.WH_KEYBOARD_LL, keyboardHook, hMod, 0);
}
}

When I return a new LRESULT(1) at the end of my proc (commented out code at the end), all keyboard events are consumed. However, when i replace it with

return lib.CallNextHookEx(hhk, nCode, wParam, info.getPointer());

as it should be, and only try to consume the main keyboard events i want to consume, it doesn't consume any of the keyboard events. Does anyone have any idea of why it won't let me consume the events I want or have any idea how to fix it so it will?

Answer 1

In order to ensure that a key is "consumed", you need to ensure that you avoid calling the next hook (i.e. return LRESULT(1)) on all event variants of a given key, i.e. WM_KEYUP, WM_KEYDOWN, and possibly WM_CHAR.

Some applications may look for key up events, others for key down, and others simply for the produced character output, so you must consume all events related to a given keystroke to make said keystroke properly "disappear".