Step by step object detection with ORB

Question

I must create an Android app that recognizes some objects from the camera (car steering wheel, car wheel). I tried with Haar classifier but without success and I'm running out of time (it's a school project). So I decided to look for another way. I found some other methods for my goal - ORB. I found what should I do in this answer. My problem is that things are messed up in my head. Can you give me a step-by-step answer of what to do to implement the answer from the question in the link I gave:

From extracting the feature points to training the KD tree and using it for every frame from the camera.

Bonus questions: Can you give a definition of feature point? It's something I couldn't exactly understand. Will be the detecting slow using ORB? I know OpenCV can be used in native android, wouldn't that make the things faster?

I need to create this app as soon as possible. Please help!

Answer 1

I am currently developing a similar application. I would recommend getting something working with a single reference image first for a couple of reasons:

1. It's easier to do and understand if you're just starting out, and you can change it later.
2. For android applications you have limited processing capabilities so more images = lower fps.

You should have a look at the OpenCV tutorials which are quite helpful. Once you go through the “OpenCV for Android SDK” section and understand the three tutorials you can pretty easily add in functionality that will allow you to analyse the video feed.

The basic logic path I'd recommend following when making the app is:

1. Read in the reference image.
2. Create and use your FeatureDetector, DescriptorExtractor and DescriptorMatcher.
3. Use the above to detect keypoints and then descrive keypoints (the first two, don't forget to convert it to a mat and then to greyscale).
4. Every time you get a frame from your camera repeat step 3. on it and then compare the keypoints in the images (with the third part of 2.).
5. Use the result to determine if there is a match (if there is then draw a box around it or something).
6. Get a new frame.

Try making it to work for a single object and then add in others later. Another thing you could add is a screen at the start to allow users to pick what they want to search for.

Also ORB is reasonably fast, especially compared to SIFT and SURF. I get about 3fps on a HTC One with a single reference image.