I am currently working on a software to track real-world objects and place an equivalent 3D version on top of it. For the past 6 months i have extensively researched and developed multiple versions of the software but i am now facing a problem i cannot solve nor find proper articles about it.
To summarize, the current state of the software is able to detect objects similar to what is done in this article https://docs.opencv.org/3.1.0/dc/d2c/tutorial_real_time_pose.html#gsc.tab=0
Which uses a combination of ORB pattern detection and SolvePnP to extract the object's pose from image frame key features. But the problem is that it only works for planar objects, anything that is more complex, the software no longer can extract the correct pose and alignment.
I tried the article's method of getting the pose of the object, with the limitation of only working with planar objects. I also tried going with an ICP (iterative closest point) approach but point-clouds are only generated with the camera movement and i need a way of tracking with a fixed camera and moving object. I have started to look into POSIT but i don't know if it's a good option.
What am i missing in terms of implementations, how can i achieve complex geometry / non-planar pose tracking taking into consideration the article mentioned? And how would one keep the pose from one angle to another (360 view of an object)? I would really appreciate some guidance in this matter!
The ultimate goal would be to recreate as close as possible this type of object tracking https://www.youtube.com/watch?v=D60Q18lMTz8
Is solvepnp the correct path?
Cheers!