How to perform Other app list item click using accessibility service like Voice Access app?

Question

I am developing an application like VoiceAccess app. Using accessibility service I am able to perform all clicks which are on top activity(3rd party applications). But I am facing issue with ListItem clicks. I am trying this code for FaceBook app. below is my code. Can any one help me on this.

public class MyService extends AccessibilityService {

/**/
private SharedPreferences S_PREF;
private SharedPreferences.Editor editor;

private static final String TAG = MyService.class
        .getSimpleName();

@Override
public void onAccessibilityEvent(AccessibilityEvent event) {
    clickPerform(getRootInActiveWindow(), 0);
}

public void clickPerform(AccessibilityNodeInfo nodeInfo, final int depth) {

    if (nodeInfo == null) return;

    List<AccessibilityNodeInfo> list = nodeInfo
            .findAccessibilityNodeInfosByViewId("com.facebook.katana:id/bookmarks_tab");
    for (AccessibilityNodeInfo node : list) {

        Log.i(TAG, "ViewID-: bookmarks_tab " + node.getChild(0));

        if (S_PREF.getBoolean("fb_menu", false)) {
            editor.putBoolean("fb_menu", false);
            node.performAction(AccessibilityNodeInfo.ACTION_CLICK);
        }
    }
    List<AccessibilityNodeInfo> list2 = nodeInfo
            .findAccessibilityNodeInfosByViewId("com.facebook.katana:id/bookmarks_list");
    for (AccessibilityNodeInfo node2 : list2) {
        if (node2.getChild(0) != null)
        if (S_PREF.getBoolean("fb_scroll_down", false)) {
            editor.putBoolean("fb_scroll_down", false);
            node2.performAction(AccessibilityNodeInfo.ACTION_SCROLL_FORWARD);
        }
    }
    editor.commit();
    for (int i = 0; i < nodeInfo.getChildCount(); ++i) {
        clickPerform(nodeInfo.getChild(i), depth+1);
    }
}



@Override
public void onInterrupt() {

}

protected void onServiceConnected() {
    S_PREF = getSharedPreferences("S_PREF", Context.MODE_PRIVATE);
    editor = S_PREF.edit();
    AccessibilityServiceInfo info = new AccessibilityServiceInfo();
    info.flags = AccessibilityServiceInfo.DEFAULT;
    info.eventTypes = AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED | AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED;
    info.feedbackType = AccessibilityServiceInfo.FEEDBACK_GENERIC;
    setServiceInfo(info);
    Toast.makeText(getApplicationContext(), "onServiceConnected", Toast.LENGTH_SHORT).show();
     }
}

Below is the VoiceAccess screen. When user says any number, then particular item click will be performed.

I am able to get 7,8,9,10 events but from 11 onwards I am not getting list items individually. My code returning listview id only.

Thanks in advance....

Answer 1

@steveenzoleko

I got solution for this problem by myself. Here AccessibilityNodeInfo always doesn't return complete list or list size. It returns VIEWs. It may be TextView, Button etc... Every View has multiple methods like getText(), getContentDescription(), getClassName(), getChildCount(), getViewIdResourceName() etc... here Voice Access app detecting all views and giving them some numbers. For lists, using getChildCount() method in for loop we can getChildViews/listItems. Ex:

AccessibilityNodeInfo node = getRootInActiveWindow();
if(node != null) {
   for(int i = 0; i < node.getChildCount(); i++){
      AccessibilityNodeInfo childNode = node.getChild(i);
      if(childNode != null){
        Log.i("childNode", "-----getText->"+childNode.getText()+"---getContentDescription-->"+childNode.getContentDescription() );
      }
   }
}

use getText() and getContentDescription() methods to know text of textview, button, checkbox.

getClassName() method retuns the view type like TextView, Button, ImageView, CheckBox etc...

Answer 2

Maybe you can use dispatchGesture from AccessibilityService API 7.0.