opencv - cropping handwritten lines (line segmentation)

Question

I'm trying to build a handwriting recognition system using python and opencv. The recognition of the characters is not the problem but the segmentation. I have successfully :

• segmented a word into single characters
• segmented a single sentence into words in the required order.

But I couldn't segment different lines in the document. I tried sorting the contours (to avoid line segmentation and use only word segmentation) but it didnt work. I have used the following code to segment words contained in a handwritten document , but it returns the words out-of-order(it returns words in left-to-right sorted manner) :

import cv2
import numpy as np
#import image
image = cv2.imread('input.jpg')
#cv2.imshow('orig',image)
#cv2.waitKey(0)

#grayscale
gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)
cv2.imshow('gray',gray)
cv2.waitKey(0)

#binary
ret,thresh = cv2.threshold(gray,127,255,cv2.THRESH_BINARY_INV)
cv2.imshow('second',thresh)
cv2.waitKey(0)

#dilation
kernel = np.ones((5,5), np.uint8)
img_dilation = cv2.dilate(thresh, kernel, iterations=1)
cv2.imshow('dilated',img_dilation)
cv2.waitKey(0)

#find contours
im2,ctrs, hier = cv2.findContours(img_dilation.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

#sort contours
sorted_ctrs = sorted(ctrs, key=lambda ctr: cv2.boundingRect(ctr)[0])

for i, ctr in enumerate(sorted_ctrs):
    # Get bounding box
    x, y, w, h = cv2.boundingRect(ctr)

    # Getting ROI
    roi = image[y:y+h, x:x+w]

    # show ROI
    cv2.imshow('segment no:'+str(i),roi)
    cv2.rectangle(image,(x,y),( x + w, y + h ),(90,0,255),2)
    cv2.waitKey(0)

cv2.imshow('marked areas',image)
cv2.waitKey(0)

Please note that i am able to segment all the words here but they appear out order.Is there any way to sort these contours in order of top to bottom

OR

segment the image into separate lines so that each line can be segmented into words using above code?

Answer 1

I got the required segmentation by making a change to the above code on the line:

kernel = np.ones((5,5), np.uint8)

I changed it to :

kernel = np.ones((5,100), np.uint8)

Now i get the outputs as following This also works with handwritten text images with lines that are not perfectly horizontal:

EDIT : For getting individual characters out of a word, do the following :

1. Resize the contour containing the word using the code as follows.

   im = cv2.resize(image,None,fx=4, fy=4, interpolation = cv2.INTER_CUBIC)
   

2. Apply same contour detection process as in line segmentation, but with a kernel of size (5,5), i.e :

   kernel = np.ones((5,5), np.uint8)
   img_dilation = cv2.dilate(im_th, kernel, iterations=1)