I am trying to create a function that will open files up to 20MB from a URL into memory. I need to create a consistent hash.
This is be closest I could get.
import os, hashlib, optparse, requests
def get_remote_sha_sum(url):
url_file = requests.get(url)
sha1 = hashlib.sha1()
with open(url_file, "rb") as f:
while True:
data = f.read(65536)
if not data:
break
sha1.update(data)
return sha1.hexdigest()
if __name__ == '__main__':
opt = optparse.OptionParser()
opt.add_option('--url', '-u', default='https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf')
options, args = opt.parse_args()
print get_remote_sha_sum(options.url)
But it results in:
TypeError: coercing to Unicode: need string or buffer, Response found
I have tried dozens of things.'I went down the path of using BitesIO
in which I am met with the same error message.
How do I open a large file in memory, buffer it, and create a hash?
Pls be kind, I'm still a little new to Python.