Image stabilization with uC/DSP

Question

I'm in the research phase for a project that involves stabilizing the image in a video stream that is going to be sent by RF. I want the processing to be done with a DSP or if possible with a uC that incorporates a DSP (like STM32F4x) near the camera (not on the receiving end).

Any idea on which algorithms would be best suited for exactly this kind of application (obviously real-time constraints are the most important)? Also, any hint on what would be best: uC or DSP? And which type?

Thanks

Answer 1

About the target :

• a DSP will be much more efficient when it's focus on repetitive and short algorithm, this seems to be your case.
• A uC will be more focused on multiple general purpose features, like communication, small computation, debug ...

Remember that most DSP are really close to uC environement : peripherals, multiple communication channel, OS ready ...

Answer 2

The best is highly subjective, however, you need to make sure you have enough memory to buffer a frame or more (depending on the algorithm) and of course some DSP features would be nice. As for the algorithm, check those questions and their answers:

Image stabilization/alignment algorithm

Video Stabilization with OpenCV

Answer 3

1. Use a microcontroller, if you are building a toy.
2. Use DSP application processor if you are building a decent consumer product
3. Use a combination of DSP + FPGA if you are building super robust product like automotive electronics/control

Choice depends on application. For pure MAC performance, I go with Analog Devices. For MAC and other bells & whistles I'd choose something from TI's video DSP portfolio. Try from them- such as DaVinci TMS320DM series. AD Blackfin used to have impressive MAC performance + uC features together. But its long term roadmap is not clear.

Complexity and detail of your problem motivate how much DSP/Math intensive your application is going to be.

1. Image, video (or image in video) stabilization can be done implicitly with the input alone or with aid of additional external measurements. It is not just DSP alone, but a combination of statistics, simple motion estimations like LSF and tracking algorithms etc. DSP is only one of the necessary collection of methods.

2. If you are building a rover/robot or car camera, a sensor fusion like 9-axis Accel + Gyro + Mag, sensor would help measure how 'bumpy' your input measurement is (camera input). These aid choosing the right coefficients on the DSP filters.

3. Also there is implicit 'motion estimation' without the help of sensors. It is called Optical Flow. The same technology used in optical mouse or drones stabilization. The rapid or slow motion of the camera gets detected based on sweep/drift amount and direction on the 2D array of pixels.

4. DSP filter, vector processing algorithms have one thing in common, repetitive multiplication and repetitive addition of results (accumulation). This needs good MAC unit, so DSP processor is the choice.

5. Even dynamically computing the right coefficients on the DSP filters is math intensive. So it has a recursive merit to use a DSP processor!