How do I select text from a scanned photo?

Question

I'm in the process of writing an app with which you can take a picture of a text and then the text is scanned and transferred to a variable. I've done that with the plugin firebase_ml_vision and everything works.

The problem I have is that I want to decide for myself which text is scanned from the photo. For example, this could work in such a way that each word and number is automatically given a frame and the user then taps the words that are transferred to the variable. This also works with Google translator (see screenshot) but unfortunately I haven't found anything yet how to do it... Do you know how it works?

Answer 1

The firebase-mlkit's text recognition API returns a frame as well as cornerPoints for each of the VisionTextBlock, VisionTextLine, and VisionTextElement:

https://firebase.google.com/docs/reference/swift/firebasemlvision/api/reference/Classes/VisionTextBlock

They should help you to select the words, lines, or text blocks.