Genfromtxt throw exceptions "got 3 columns instead of 27" bu it is not so

Question

I use:

Data = np.genfromtxt(filename, delimiter='"\t"', dtype=None, autostrip=True, skip_header=1)

And in proccess it gives me raise ValueError(errmsg)

Line #33 (got 3 columns instead of 27)

But it is not so. In file in that line I have all columns! I checked - function read those lines with "missing values" untill some symbol: For example, in line 33 function read this:

"http://www.savvyeat.com/whole-wheat-chocolate-chai-muffins/"   "2152"  "{""title"":""Whole Wheat Chocolate Chai Muffins Savvy Eats "",""body"":""I think I subconsciously sabotaged myself Two weeks ago I couldn t

How can I read my file in numpy array in other way or somehow fix this problem?

Answer 1

This kind of problem should be quickly solvable once we can see the contents of the CSV file. To debug the problem run:

import itertools as IT
with open(filename, 'rb') as f:
    content = ''.join(IT.islice(f, 50))
    print(repr(content))

and post the output. That will give us the first 50 lines of the file. If there is sensitive data, you can redact it before posting; just leave the quotation marks and \t intact.