How can I make CefSharp.Wpf.HwndHost open the touch keyboard

Question

We've historically used CefSharp.Wpf + Touch keyboard sample to open the tablet keyboard on a WPF Kiosk that wraps CEF Sharp. Since it is using CefSharp.Wpf - we are facing the relatively common GPU rendering issues on certain devices, as well as disappointment with the performance. Switching to CefSharp.Wpf.HwndHost works to solve our GPU issues, but doesn't reliably trigger the touch keyboard.

How can I trigger the Windows 10 touch keyboard? (OSK / TabTip.exe)

Things that don't work:

• disable-usb-keyboard-detect does not by itself produce reliable SIP activation
• CefSharp.Wpf.HwndHost by itself does not produce reliable SIP activation, but does sometimes work and does produce correct SIP type for numeric et al.
• CefSharp.Wpf does not produce any SIP activation (in box), and produces GPU artifacts and performance issues on many machines
• CefSharp.Wpf + Touch keyboard sample does produce reliable SIP activation, but does not control SIP keyboard type (e.g.: numeric pad not shown on <input type="number") and still suffers from GPU and performance issues

Answer 1

You can use the same IInputPane2 interface to trigger the touch keyboard. CefSharp.Wpf.HwndHost requires use of IRenderProcessMessageHandler to receive focus changes, since the VirtualKeyboardRequested event is specific to CefSharp.Wpf.

Since the IRenderProcessMessageHandler.OnFocusedNodeChanged event fires on a fairly high frequency and in cases where no OSK is desired, it is required to filter and debounce the event. It is trivial to do this using Rx.Net. A full example can be found on Github.

Project:

• Set TFM to net6.0-windows10.0.19041
• Add <PackageReference Include="CefSharp.Wpf.HwndHost" Version="98.1.210" />
• Add <PackageReference Include="System.Reactive" Version="5.0.0" />

Program.cs:

Cef.EnableHighDPISupport();

var settings = new CefSettings();
CefSharpSettings.FocusedNodeChangedEnabled = true;
settings.CefCommandLineArgs.Add("disable-usb-keyboard-detect", "1");
Cef.Initialize(settings);

MainWindow.xaml.cs:

public partial class MainWindow : Window
{
    private Lazy<(IInputPaneInterop ipi, IInputPane2 ip)> sip;

    public MainWindow()
    {
        InitializeComponent();

        sip = new Lazy<(IInputPaneInterop ipi, IInputPane2 ip)>(() =>
        {
            var hwnd = new WindowInteropHelper(this).Handle;
            var ipi = InputPane.As<IInputPaneInterop>();
            var ip = ipi.GetForWindow(hwnd, typeof(IInputPane2).GUID);
            return (ipi, ip);
        });

        var oskSubject = new Subject<bool>();
        cwb.RenderProcessMessageHandler = new OskRenderProcessMessageHandler(oskSubject.OnNext);
        oskSubject
            .Throttle(TimeSpan.FromMilliseconds(200))
            .ObserveOn(SynchronizationContext.Current ?? throw new InvalidOperationException("No syncctx"))
            .Subscribe(PopOsk);
    }

    protected override void OnClosed(EventArgs e)
    {
        if (sip.IsValueCreated)
        {
            var (ipi, ip) = sip.Value;
            Marshal.FinalReleaseComObject(ip);
            Marshal.FinalReleaseComObject(ipi);
        }
        base.OnClosed(e);
    }

    private void PopOsk(bool shouldShow)
    {
        var (_, ip) = sip.Value;
        if (shouldShow)
        {
            Debug.WriteLine($"Showing SIP");
            ip.TryShow();
        }
        else
        {
            Debug.WriteLine($"Hiding SIP");
            ip.TryHide();
        }
    }
}

internal class OskRenderProcessMessageHandler : IRenderProcessMessageHandler
{
    private readonly Action<bool> SetOsk;

    public OskRenderProcessMessageHandler(Action<bool> popOsk)
    {
        this.SetOsk = popOsk;
    }

    public void OnContextCreated(IWebBrowser chromiumWebBrowser, IBrowser browser, IFrame frame)
    {
        // nop
    }

    public void OnContextReleased(IWebBrowser chromiumWebBrowser, IBrowser browser, IFrame frame)
    {
        // nop
    }

    public void OnFocusedNodeChanged(IWebBrowser chromiumWebBrowser, IBrowser browser, IFrame frame, IDomNode? node)
    {
        SetOsk(node != null && "input".Equals(node.TagName, StringComparison.InvariantCultureIgnoreCase));
    }

    public void OnUncaughtException(IWebBrowser chromiumWebBrowser, IBrowser browser, IFrame frame, JavascriptException exception)
    {
        // nop
    }
}

[ComImport, Guid("75CF2C57-9195-4931-8332-F0B409E916AF"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IInputPaneInterop
{
    void _VtblGap1_3();

    IInputPane2 GetForWindow([In] IntPtr appWindow, [In] ref Guid riid);
}

[ComImport, Guid("8A6B3F26-7090-4793-944C-C3F2CDE26276"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
internal interface IInputPane2
{
    void _VtblGap1_3();

    bool TryShow();
    bool TryHide();
}

Error checking is elided for expository purposes.

General considerations:

1. Only tested with Windows 10 Enterprise 20H2 (10.0.19042) and higher. Not tested on W11
2. No hardware keyboard should be present (or GPIO should indicate that the keyboard is disabled - ex: Lenovo Yoga in tablet mode)
3. System must be configured to use touch keyboard in desktop mode (unclear on requirement)
4. Behavior varies based on touch input device and system state when the app is started
5. Behavior varies based on BorderStyle
6. Triggering gesture must originate be touch - cannot use mouse to test