iOS 13 Voice Control Accessibility Feature - is there a way to do more custom actions (rather than just linking to accessibility labels)?

Question

The new Voice Control API is super cool, it's the one where you can speak to your iOS device and perform actions. (Video from Apple)

My only gripe is that finding information on it is kinda difficult. I don't see any WWDC videos out there on it, and I can't find any other documentation.

It's basically powered by accessibilityLabels. Since each accessibilityElement can really only have one accessibilityLabel it's (from what I can see) limited to that.

Is that accurate? Is there a way to provide users with more custom actions? For instance there's the accessibility custom actions API that allows you to add more by swiping up/down with VoiceOver, but those don't seem to be available in any way to Voice Control, it's just the accessibilityLabel.

It's such a cool API, but with VoiceOver custom actions and rotor actions I can normally provide more easily accessible actions to users, and I can't figure out how to do that for a user who uses Voice Control.

Answer 1

It's basically powered by accessibilityLabels.

Nothing is highlighted about this amazing new feature but its 'accessibilityUserInputLabels' property that enables changing labels specifically for some items: neither event name nor notification are exposed unfortunately.

Labels represent an interesting way of choosing items vocally but I'm not sure that they will be the most used by users: displaying items with numbers seems to be more efficient in certain circumstances (VoiceControl ➔ show numbers).

The official release of iOS 13 should provide some news in the final documentation.

Is there a way to provide users with more custom actions? For instance there's the accessibility custom actions API that allows you to add more by swiping up/down with VoiceOver, but those don't seem to be available in any way to Voice Control, it's just the accessibilityLabel.

The interest of this new feature is affording people to vocally use their devices as they do with touches, that's why there's no need of having specific actions dedicated to VoiceControl: all possible actions are those available with touches.

However, you can reach the existing item actions saying show actions for YourItemNumber to get an action sheet showing its actions.

Besides, you can create new vocal commands that could be similar to the VoiceOver custom actions thanks to the Customize Commands - Create New Command... menu where all the vocal commands are available as well: (nothing is currently code reachable)

If custom actions mean new vocal commands, that's the rationale to be followed but if you want to add new graphical command only for the Voice Control feature, that's currently not possible under iOS 13.