Using Windows Tablet PC Input to implement handwriting recognition

Question

I want to write a app (initially Windows) that include handwriting to text recognition. I want to use the Windows built-in Tablet PC INput. My question is is there a way to capture the strokes as an image, "send these to the OCR engine used by the Tablet Input, and return the recognised text?

Or, are there any good open source handwriting libraries that could be used directly?

The primary development language is Qt.

Answer 1

I am not aware of any open source or free software libraries for handwriting recognition, so I wrote an adapter. My target was my tablet PC running Linux, but part of my solution can also be used directly on Windows, although you will need to adapt it to your needs.

You will need to read through the licenses for the components I used and validate your own use of them.

The source is available here: Ink2Text project

Part of this solution is a server which uses the XP Handwriting Recognition libraries to interpret the strokes which make up handwriting. As an aside, this does not use OCR - it uses connected graphs of the flow of the strokes.

Another complementary project provides a client handwriting widget: Stylus/Handwriting Input Panel. This is written in Java, and it's GPL3. It accepts the handwriting and sends it off to the server. Unless you wish to use it as is, it's of value solely to see the data format for the ink, although that's simple enough and you can probably deduce that with just the Ink2Text source code.

An earlier solution used the S/HIP with my MS Ink Server, which accepted input over regular network connections. That may also be useful depending on your architecture, but requires a running copy of Windows.

This system provides very good recognition of printed and cursive handwriting.

I will answer questions about it only in it's associated SourceForge forums, so that others may benefit from the answers as well - please don't ask here.

Cheers, Bret

Answer 2

I want to be wrong, but unfortunately, there is no available open-source offline handwriting recognition system even close to MS' or Apple's Ink.

On Windows you can play with Ink Recognition (About Handwriting Recognition, Advanced Recognition Sample). C++ interface is available, but not as well documented, as .net implementation is. So, you need to apply more efforts and do a lot of research to achieve what you want.

For another systems (including Windows too) there is way to use Tesseract-OCR with your application. See Tesseract's base api. For better recognition quality, you may train tesseract and use your own trained data.

If you do not want to spend your time doing R&D tasks above, you can use paid solutions like: MyScript SDK, WritePad SDK and so on...