I am using bottlenose to make requests from Amazon and then parsing the response with BeautifulSoup.
I have the exact same python script on my Mac and also on a Raspberry-Pi.
#!/usr/bin/python
import sys
import ids
import csv
import bottlenose as BN
import lxml
import datetime
import os
from bs4 import BeautifulSoup
import numpy
import time
from urllib2 import HTTPError
def error_handler(err):
ex = err['exception']
if isinstance(ex, HTTPError) and ex.code == 503:
time.sleep(random.expovariate(0.1))
return True
amazon = BN.Amazon(AWSKEY,AWSSECRET,AWSID,Region='UK', MaxQPS=0.8, ErrorHandler=error_handler)
isbnclean = '0521663962'
response = amazon.ItemLookup(ItemId=isbnclean, ResponseGroup="Large")
print response
print 'Test Done'
So this script will run perfectly fine on my mac and print the response, but on the R-Pi returns:
File "SimpleTest.py", line 24, in <module>
response = amazon.ItemLookup(ItemId=isbnclean, ResponseGroup="Large")
File "/usr/local/lib/python2.7/dist-packages/bottlenose/api.py", line 265, in __call__
{'api_url': api_url, 'cache_url': cache_url})
File "/usr/local/lib/python2.7/dist-packages/bottlenose/api.py", line 226, in _call_api
return urllib2.urlopen(api_request, timeout=self.Timeout)
File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 475, in error
return self._call_chain(*args)
File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 558, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 400: Bad Request
The only thing that differs is that on the R-Pi, the from urllib2 import HTTPError
is essential whereas on the mac it will run without.
I am unsure where this is going wrong, as I used to be able to run the script on the R-Pi perfectly fine.
AWS will flag up 400 Bad request errors for multiple reasons. In my case, the raspberry Pi timestamp was out of sync with true time for some reason and as such the requests to amazon were stopped.
You can read more here AWS Error Codes