Remove lens distortion from images captured by an wide angle (180) camera

Question

I have some images captured from an wide angle appx. (180 degree) camera. I am using opencv 2.4.8 which gives some details about camera matrix n distortion matrix.

MatK = [537.43775285, 0, 327.61133999], [0, 536.95118778, 248.89561998], [0, 0, 1]
MatD = [-0.29741743, 0.14930169, 0, 0, 0]

And this info I have used further to remove the distortion. But the result is not as expected. I have attached some input images of chess board which i have used to calibrate. Or Is there any other tools or library by which it can be removed.

input images

from a Normal Camera or even captured by my smart phone

Answer 1

The most common method (used by the Oculus Rift Runtime for example) draws a fine enough textured grid for which the texture coordinates or the grid node positions are chosen to compensate the distortion. To obtain the grid normally one fits a polynomial or a spline to some reference picture. For example the checkerboard in your camera is a common calibration target.

Answer 2

The whole point of the calibration process is to tell OpenCV what a straight line looks like under distortion. A chess board is used to present a number of straight lines that are easy for OpenCV to detect. In your image, these lines are simply not straight. I'm moderately sure that OpenCV also needs square boxes.

So, use a real chess board pattern. Print it out, glue it to a piece of wood or hard plastic or whatever. But make sure it's a regular chessboard pattern on a level plane.

Answer 3

This is not an answer to the question, but something about the "discussion" of distortion and planarness.

In reality you have some straight lines on a pattern:

With (nearly any) lens you'll get some kind of distortion so that those straight lines aren't straight anymore after projection to your image. This effect is much stronger for wide angle lenses. You could expect something like this (for wide angle stronger but similar):

But the images you provided look more like this, which can be because of your pattern wasnt really planar on the ground, or because the lens has some additional "hills" on your lens.