How to extract tables from a pdf with PDFMiner?

Question

I am trying to extract information from some tables in a pdf document.
Consider the input:

Title 1
some text some text some text some text some text
some text some text some text some text some text

Table Title
| Col1          | Col2    | Col3    |
|---------------|---------|---------|
| val11         | val12   | val13   |
| val21         | val22   | val23   |
| val31         | val32   | val33   |

Title 2
some more text some more text some more text some more text
some more text
some more text some more text some more text some more text

I can get the outlines/titles as such:

path='myFile.pdf'
# Open a PDF file.
fp = open(path, 'rb')
# Create a PDF parser object associated with the file object.
parser = PDFParser(fp)
# Create a PDF document object that stores the document structure.
# Supply the password for initialization.
document = PDFDocument(parser, '')
outlines = document.get_outlines()
for (level,title,dest,a,se) in outlines:
    print (level, title)

This gives me:

(1, u'Title 1')
(2, u'Table Title')
(1, u'Title 2')

Which is perfect, as the levels are aligned with the text hierarchy. Now I can extract the text as follows:

if not document.is_extractable:
    raise PDFTextExtractionNotAllowed
# Create a PDF resource manager object that stores shared resources.
rsrcmgr = PDFResourceManager()
# Create a PDF device object.
laparams = LAParams()
device = PDFPageAggregator(rsrcmgr, laparams=laparams)
# Create a PDF interpreter object.
interpreter = PDFPageInterpreter(rsrcmgr, device)
# Process each page contained in the document.
text_from_pdf = open('textFromPdf.txt','w')
for page in PDFPage.create_pages(document):
    interpreter.process_page(page)
    layout = device.get_result()
    for element in layout:
        if isinstance(element, LTTextBox):
            text_from_pdf.write(''.join([i if ord(i) < 128 else ' '
                                            for i in element.get_text()]))

Which gives me:

Title 1
some text some text some text some text some text some text some text
some text some text some text some text some text some text some text
Table Title
Col1
val11
val12
val13
Col2
val21
val22
val23
Col3
val31
val32
val33
Title 2
some more text some more text some more text some more text
some more text
some more text some more text some more text some more text

Which is a bit weird as the table is extracted in a column-wise fashion. Would it be possible for me to get the table row by row? Moreover, how can I identify where a table begins and ends?

Answer 1

Use camelot for extracting tables from pdfs

Answer 2

If you only want to extract tables from PDF documents, then look at this answer: How to extract table as text from the PDF using Python?

From that answer, I have tried tabula-py which worked for me with tables of figures spread over multi-page PDF. tabula-py skipped properly all the headers and footers. Previously I had tried PDFMiner on this same type of document, and I had the same problem you mentioned, and sometimes even worse.