huge dot pattern on text removal

Question

I am trying to extract the text so that it can we OCR processed but these dots add a lot of noise. Image: http://img22.imageshack.us/img22/1344/l0ap.png

Thanks in Advance!

Answer 1

I thought this looked like an interesting problem that MSER blob detection and inpainting could solve. Below is some code I tried; but I don't think the result is acceptable for OCR input; but, I've included it anyway incase it might prove useful. The inpainting didn't extend the contour lines of the characters into the mask region in the manner I was hoping it would. I think a more promising approach would be as follows

• compute image gradient magnitude
• threshold gradients
• consider points above the threshold as candidates for curve points
• trace curves with a upper curvature bound that is below the minimum curvature of the dots (say 3-6 standard deviations below the mean curvature of the dots), that way no candidate curves will be generated on the boundaries of the dots, only the exposed segments of the letters. This is based on the observation that dots have high curvature and character boundary segments have low curvature between corners.
• extrapolate and intersect candidate curves with low order polynomials.
• detect and connect cycles of curve segments
• rasterize any closed loops of curves with filled black on a white background
• pass to OCR.

MSER+inpainting attempt:

//Find blobs, assuming the image to repair is in a cv::Mat text
cv::Mat grey;
cv::cvtColor(text, grey, CV_RGB2GRAY);
//MSER has an easier time finding these dots if there are more pixels to work with
cv::resize(grey, grey, cv::Size(text.cols*2, text.rows*2), cv::INTER_CUBIC);
cv::blur(grey, grey, cv::Size(3, 3));
int delta = 1;
int minPixels = 5;
int maxPixels = 400;
float maxVariation = 0.4;
float minDiversity = 0.1;
cv::MSER detector(delta, minPixels, maxPixels, maxVariation, minDiversity);
std::vector<std::vector<cv::Point> > blobs;
detector(grey, blobs);

//Find the radius of each blob
cv::Mat radii((int)blobs.size(), 1, CV_64F);
for (int i = 0; i < blobs.size(); i++)
{
    cv::Point2f center; //not used
    float rad;
    cv::minEnclosingCircle(blobs[i], center, rad);
    radii.at<double>(i, 0) = (double)rad;
}

//Build a Gaussian mixture histogram
cv::TermCriteria criteria;
criteria.maxCount = 500;
criteria.type = cv::TermCriteria::COUNT;
cv::EM model = cv::EM(4, cv::EM::COV_MAT_DIAGONAL, criteria);
model.train(radii);

//Get the stats for each Gauss peak in the Gaussian mixture model
cv::Mat weights = model.get<cv::Mat>("weights");
cv::Mat means = model.get<cv::Mat>("means");
vector<cv::Mat> covs = model.get< vector<cv::Mat> >("covs");


//Identify the heaviest peak to use as the classifier for dots
float heaviestPeakWeight = 0;
int heaviestPeakId = -1;
for (int i = 0; i < weights.size().width; i++)
{
    if (weights.at<double>(0, i) > heaviestPeakWeight)
    {
        heaviestPeakWeight = weights.at<double>(0, i);
        heaviestPeakId = i;
    }
}

//Classify the blobs by their radius, we make the assumption
//that because the dots are more numerous than other features
//and that their size is uniform, they should cause a sharp
//peak in the histogram
const double Sqrt2Pi = sqrt(2*M_PI);
std::vector<int> blobsInHeaviest;
blobsInHeaviest.reserve(blobs.size());
for (int i = 0; i < radii.rows; i++)
{
    //For each radius find the strongest Gauss peak it lies under
    double maxClassVal = 0;
    int maxClassId = -1;
    double x = radii.at<double>(i, 0);
    for (int j = 0; j < weights.size().width; j++)
    {
        double mean = means.at<double>(0, j);
        double variance = covs[j].at<double>(0, 0);
        double weight = weights.at<double>(0, j);

        double classVal =   weight*exp(-pow((x - mean), 2)/(2*variance))/
                            (Sqrt2Pi*variance);
        if (classVal >= maxClassVal)
        {
            maxClassVal = classVal;
            maxClassId = j;
        }
    }
    if (maxClassId == heaviestPeakId)
    {
        blobsInHeaviest.push_back(i);
    }
}

//Rasterize the blobs to create an inpaint mask to remove the dots
cv::Mat mask(grey.size(), CV_8UC1, cv::Scalar(0));
for(int i = 0; i < blobsInHeaviest.size(); i++)
{
    for (int j = 0; j < blobs[blobsInHeaviest[i]].size(); j++)
    {
        mask.at<uchar>(blobs[blobsInHeaviest[i]][j]) = 255;
    }
}

//Prior to inpainting we need to ensure the mask components cover
//the white lines between the dots and the letters. MSER makes them
//a bit undersized so we can use morphological dilate to make them
//a little larger.
double scale = means.at<double>(0, heaviestPeakId);
cv::dilate(mask, mask,
           cv::getStructuringElement(cv::MORPH_RECT,
                                     cv::Size(rint(3*scale),rint(3*scale))));
cv::inpaint(grey, mask, grey, rint(1.7*scale), cv::INPAINT_NS);

//Dynamic thresholding could be accomplished by exploiting the assumption
//that the histogram for an image of text will have a step like histogram
//use a step detection algorithm and threshold at the step location. I
//did not do this because thresholding is not the problem with this approach.
cv::threshold(grey, grey, 130, 255, cv::THRESH_BINARY);
cv::resize(grey, grey, cv::Size(text.cols/2, text.rows/2));
//Finished result is in grey

Edit: mixed up curvature and bend radius.

Answer 2

You already have a great solution here, but I still want to add another approach.

1- binary threshold with a very low value

2- find all the contours and list their areas. Fill small contours with white.

3- try OCR, if it doesnt give you a numerical answer, add more preprocessing as such:

while(!ocr)    
{    
do morphological closing,
do morphological opening,
fill small blobs with white,
try ocr.
}

morphological operations will help you to cut small limbs(dots) out of your blobs (numbers).