I am trying to decompress a big gzip file (2GB). If I use the read() function, it freezes my computer due to so much of data. So, looking at another answer on stackoverflow, I used the approach of reading the gzip file line by line
inF = gzip.open(compressed_file_name, 'rb')
outF = open(outfilename, 'wb')
for line in inF:
outF.write(line)
inF.close()
outF.close()
However, at the end of the file, I get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 12, in download_latest_file
File "/usr/lib/python2.7/gzip.py", line 455, in readline
c = self.read(readsize)
File "/usr/lib/python2.7/gzip.py", line 261, in read
self._read(readsize)
File "/usr/lib/python2.7/gzip.py", line 308, in _read
self._read_eof()
File "/usr/lib/python2.7/gzip.py", line 347, in _read_eof
hex(self.crc)))
IOError: CRC check failed 0x3b5a517c != 0x643d9301L
My python version is 2.7.6