return stream of url file with python urllib2

Question

I've got a piece of Python code that gets a url with the right permissions from Amazon S3 and copies it to a local directory file:

def fileDownload(self, some_Id, localDir, name, range=None):
    # [...] something happens here
    # get the Amazon URL 
    fileUrl = we_get_the_amazon_url_here
    req = urllib2.urlopen(fileUrl)
    if len(range):
        req.headers['Range']='bytes=%s-%s' % (range[0], range[1])

    # Do the download
    with open(os.path.join(localDir,name), 'wb') as fp:
        shutil.copyfileobj(req, fp)
    return 1

I am unfamiliar with urllib2 but what I would like to do is to turn this fileDownload method into a fileStreaming method, so that I can pipe the resulting content of the file into a tool downstream.

Any ideas how to do this?

Answer 1

I think you should read some documentation. However your "req" object is file-like, so you can use read() method to get its contents.

def fileDownload(self, some_Id, range=None):
    # [...] something happens here
    # get the Amazon URL 
    fileUrl = we_get_the_amazon_url_here
    req = urllib2.urlopen(fileUrl)
    if len(range):
        req.headers['Range']='bytes=%s-%s' % (range[0], range[1])

    return req.read()