Full screen TYPE_ACCESSIBILITY_OVERLAY

Question

I'm new to the accessibility stuff on Android. While going through the classes and documentation I came across TYPE_ACCESSIBILITY_OVERLAY inside the WindowManager class.

The documentation says (only the relevant text)

For example, if there is a full screen accessibility overlay that is touchable, the windows below it will be introspectable by an accessibility service even though they are covered by a touchable window.

So I set out to achieve just that, a full screen accessibility overlay and try to introspect the windows below it

Extended AccessibilityService and added my full screen overlay when onServiceConnected is called (the inspiration for adding overlay came from here)

@Override
protected void onServiceConnected() {
    WindowManager wm = (WindowManager) getSystemService(WINDOW_SERVICE);
    FrameLayout mLayout = new FrameLayout(this);
    WindowManager.LayoutParams lp = new WindowManager.LayoutParams();
    lp.type = WindowManager.LayoutParams.TYPE_ACCESSIBILITY_OVERLAY;
    lp.format = PixelFormat.TRANSLUCENT;
    lp.flags |= WindowManager.LayoutParams.FLAG_FULLSCREEN;
    lp.width = WindowManager.LayoutParams.MATCH_PARENT;
    lp.height = WindowManager.LayoutParams.MATCH_PARENT;
    lp.gravity = Gravity.TOP;
    wm.addView(mLayout, lp);

    mLayout.setOnTouchListener(new View.OnTouchListener() {
        @Override
        public boolean onTouch(View v, MotionEvent event) {
            // Here I'm getting the touch events on the overlay I added
            return false;
        }
    });
}

Now, the question is, how do I introspect or find the windows below this overlay? Even in the onAccessibilityEvent callback I get just this overlay window. getWindows() always has a size of 1. Doesn't it refute the assertion made above for TYPE_ACCESSIBILITY_OVERLAY?

Relevant info: To receive the touch events on the overlay I have disabled touchExplorationMode in the service settings

android:canRequestTouchExplorationMode="false"

Answer 1

What you seem to be missing is flagRetrieveInteractiveWindows on your configuration. These properties and window layout paremeters configuration should work, without requiring for you to disable canRequestTouchExplorationMode in order to get the events and having getWindows return the AccessibilityWindowInfo instances underneath yours:

    <?xml version="1.0" encoding="utf-8"?>
    <accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
        android:packageNames="test.demo.com.tests"
        android:accessibilityEventTypes="typeAllMask"
        android:accessibilityFlags="flagRetrieveInteractiveWindows|flagReportViewIds|flagIncludeNotImportantViews"
        android:accessibilityFeedbackType="feedbackAllMask"
        android:notificationTimeout="100"
        android:canRetrieveWindowContent="true"
        />

And on service connected:

    @Override
    protected void onServiceConnected() {
        WindowManager windowManager = (WindowManager) getSystemService(WINDOW_SERVICE);
        FrameLayout layout = new FrameLayout(this);

        WindowManager.LayoutParams params = new WindowManager.LayoutParams(WindowManager.LayoutParams.MATCH_PARENT,
            WindowManager.LayoutParams.MATCH_PARENT, WindowManager.LayoutParams.TYPE_ACCESSIBILITY_OVERLAY,
                WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE| WindowManager.LayoutParams.FLAG_FULLSCREEN |
                WindowManager.LayoutParams.FLAG_NOT_TOUCHABLE|
                WindowManager.LayoutParams.FLAG_DRAWS_SYSTEM_BAR_BACKGROUNDS| 
                WindowManager.LayoutParams.FLAG_WATCH_OUTSIDE_TOUCH,
            PixelFormat.TRANSLUCENT);
        params.gravity = Gravity.TOP;

        windowManager.addView(layout, params);
        layout.setOnTouchListener(new View.OnTouchListener() {
            @Override
            public boolean onTouch(View v, MotionEvent event) {
                //You can either get the information here or on onAccessibilityEvent                 
                return false;
            }
        });
    }

EDIT:

Added FLAG_DRAWS_SYSTEM_BAR_BACKGROUNDS to accomplish full screen and removed canRequestTouchExplorationMode since the flag associated to this property should not be included and, therefore, of no use.