Open file as a buffer in Python?

Question

I have a compressed HDF file (HDF.Z) and would like to open it like that:

from subprocess import Popen, PIPE
f = Popen(['zcat', 'myfile.HDF.Z'], stdout=PIPE).stdout

In order to get the data I need to use pyhdf:

from pyhdf.SD import SD, SDC
mydata = SD(f, SDC.READ)

However, this results in an error message:

*** TypeError: coercing to Unicode: need string or buffer, file found

Is there a way to open this file as a buffer to read it in? Btw: what is a buffer?

Answer 1

At a short glance, I found no way to make it access an open stream.

You can do the following:

• Create a temporary file where you unpack the file.
• Give the name of this temporary file to SD().

Another option is very system dependent: you could take the file handle and do

SD('/dev/fd/%d' % f.fileno(), SDC.READ)

bit this is very platform-dependent (Linux only) and, if SD() does mmap() by any chance, it will fail.