VNDocumentCameraScan to searchable PDF

Question

I am currently trying to create a searchable PDFDocument after I have obtained a VNDocumentCameraScan with the help of the VNDocumentCameraViewController.

Currently I only take the images of the scan and put them into a PDFDocument instance.

func documentCameraViewController(_ controller: VNDocumentCameraViewController, didFinishWith scan: VNDocumentCameraScan) {
   let pdf = createPDF(from: scan)
} 

fileprivate func createPDF(from scan: VNDocumentCameraScan) -> PDFDocument {
    let pdfDocument = PDFDocument()
    for i in 0 ..< scan.pageCount {
        let pdfPage = PDFPage(image: scan.imageOfPage(at: i))
            pdfDocument.insert(pdfPage!, at: i)
        }
    return pdfDocument
}

I also know how I would extract text out of the VNDocumentCameraScan. The thing that I miss is how I incorporate the text information into the PDFDocument instance. I need this because I want to scan documents, save them as .pdf to the file system and search them afterwards.

I searched a lot but did not find a way to do that.

Does anyone know how I would accomplish this ?

Answer 1

This blog post covers the topic in detail. I'm linking to Part 3 of a series, since it addresses the part of the process you're stuck on.

https://alexanderweiss.dev/blog/2021-03-29-from-uiimage-to-searchable-pdf-part-3

The main idea is to draw the recognized text underneath the image in your pdf. Basic steps from the article:

1. Take the image
2. Recognize the text on the image
3. Create a PDF page with the dimensions of the image
4. Use the recognized text to draw text
5. Draw the image above the text on the pdf

VNRecognizedText contains information about the location of recognized text, which allows you to determine where to draw the text. I was able to use the code from this post successfully in my app; although it's not 100% perfect when dealing with handwritten text, it works reasonably well.