python tarfile error: struct.error: unpack requires a string argument of length 4

Question

I have a bunch of files on which I do the following processing :

for file_name in filelist:
  tar_file = tarfile.open(file_name)
  tar_file.extractall("./" + "location")

For a particular file I am getting this error immediately after the tarfile.open call:

$ file file_name.tgz 
file_name.tgz: gzip compressed data, from Unix, last modified: Mon Dec 16 16:33:34 2013

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/tarfile.py", line 1660, in open
    return func(name, "r", fileobj, **kwargs)
  File "/usr/lib/python2.7/tarfile.py", line 1727, in gzopen
    **kwargs)
  File "/usr/lib/python2.7/tarfile.py", line 1705, in taropen
    return cls(name, mode, fileobj, **kwargs)
  File "/usr/lib/python2.7/tarfile.py", line 1574, in __init__
    self.firstmember = self.next()
  File "/usr/lib/python2.7/tarfile.py", line 2319, in next
    tarinfo = self.tarinfo.fromtarfile(self)
  File "/usr/lib/python2.7/tarfile.py", line 1239, in fromtarfile
    buf = tarfile.fileobj.read(BLOCKSIZE)
  File "/usr/lib/python2.7/gzip.py", line 256, in read
    self._read(readsize)
  File "/usr/lib/python2.7/gzip.py", line 320, in _read
    self._read_eof()
  File "/usr/lib/python2.7/gzip.py", line 339, in _read_eof
    isize = read32(self.fileobj)  # may exceed 2GB
  File "/usr/lib/python2.7/gzip.py", line 25, in read32
    return struct.unpack("<I", input.read(4))[0]
struct.error: unpack requires a string argument of length 4

Question: How can I avoid this?

Answer 1

You want some exception handling:

for file_name in filelist:
    try:
        tar_file = tarfile.open(file_name)
        tar_file.extractall("./" + "location")
    except struct.error, e:  # or except struct.error as e, depends on Python version
        print "Corrupt:", file_name
    except tarfile.TarError, e:
        print "Tar error (%s): %s" % (str(e), file_name)

This way you see the error, document it, but keep going.

http://docs.python.org/2.7/library/tarfile.html documents the exceptions from the tarfile module.