Same script on two PCs, only one gets HTTP Error 400

115 views Asked by At

I am using bottlenose to make requests from Amazon and then parsing the response with BeautifulSoup.

I have the exact same python script on my Mac and also on a Raspberry-Pi.

#!/usr/bin/python
import sys
import ids
import csv
import bottlenose as BN
import lxml
import datetime
import os
from bs4 import BeautifulSoup
import numpy
import time
from urllib2 import HTTPError


def error_handler(err):
    ex = err['exception']
    if isinstance(ex, HTTPError) and ex.code == 503:
        time.sleep(random.expovariate(0.1))
        return True

amazon = BN.Amazon(AWSKEY,AWSSECRET,AWSID,Region='UK',                 MaxQPS=0.8, ErrorHandler=error_handler)

isbnclean = '0521663962'

response = amazon.ItemLookup(ItemId=isbnclean, ResponseGroup="Large")
print response
print 'Test Done'

So this script will run perfectly fine on my mac and print the response, but on the R-Pi returns:

 File "SimpleTest.py", line 24, in <module>
    response = amazon.ItemLookup(ItemId=isbnclean,     ResponseGroup="Large")
  File "/usr/local/lib/python2.7/dist-packages/bottlenose/api.py", line 265, in __call__
    {'api_url': api_url, 'cache_url': cache_url})
  File "/usr/local/lib/python2.7/dist-packages/bottlenose/api.py", line 226, in _call_api
    return urllib2.urlopen(api_request, timeout=self.Timeout)
  File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 550, in http_response
     'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
 urllib2.HTTPError: HTTP Error 400: Bad Request

The only thing that differs is that on the R-Pi, the from urllib2 import HTTPError is essential whereas on the mac it will run without. I am unsure where this is going wrong, as I used to be able to run the script on the R-Pi perfectly fine.

1

There are 1 answers

0
Astro David On

AWS will flag up 400 Bad request errors for multiple reasons. In my case, the raspberry Pi timestamp was out of sync with true time for some reason and as such the requests to amazon were stopped.

You can read more here AWS Error Codes