Importing large integers from a mixed datatype file with numpy genfromtxt

715 views Asked by At

I have a file with the format:

1    2.5264    24106644528  astring

I would like to import the data. I am using:

>>> numpy.genfromtxt('myfile.dat',dtype=None)

Traceback (most recent call last):
  File "<pyshell#4>", line 1, in <module>
    numpy.genfromtxt('myfile.dat',skip_header=27,dtype=None)
  File "C:\Python27\lib\site-packages\numpy\lib\npyio.py", line 1691, in genfromtxt
    output = np.array(data, dtype=ddtype)
OverflowError: Python int too large to convert to C long

I checked the maximum integer on my (32-bit) system:

>>> import sys
>>> sys.maxint
2147483647

Is there a way to increase the integer limit? Or can I get around my import problem another way (without putting '.0' after all of the ints in file)?

1

There are 1 answers

4
Lee On BEST ANSWER

Realised I can do this:

>>> numpy.genfromtxt('myfile.dat',dtype=['i4','f8','f8','a14'])

array((1, 2.5264, 24106644528.0, 'astring'), 
  dtype=[('f0', '<i4'), ('f1', '<f8'), ('f2', '<f8'), ('f3', 'S14')])