python3 try/except appears to fail when using a tenacity decorator

772 views Asked by At

Trying to get tenacity to work with a requests function to implement a retry/backoff then return HTTP status if all the retries fail. It seems that I can either get the retry working or use try/except, but not both at the same time. Wrapping in another function doesn't help.

Python 3.7.7 on OSX mojave requests 2.24.0 tenacity 6.2.0

function 1 with a decorator and without try/except invokes tenacity rules, but I'm not getting the http error as a return:

In [1330]: @retry(reraise=True,wait=wait_fixed(1), stop=stop_after_attempt(3))
      ...: def geturl(url):
      ...:     """
      ...:     get the url and raise http request errors for tenacity
      ...:     """
      ...:     print(datetime.now().strftime('%Y-%m-%d-%H:%M:%S'))
      ...:     headers = {'user-agent': 'my-app/0.0.1','Content-Type':'application/json'}
      ...:     data = {'name':'testcall','service':'armor'}
      ...:     r=None
      ...:     r = requests.post(url, data=json.dumps(data), headers=headers)
      ...:     r.raise_for_status()
      ...:     return r.status_code
      ...:

In [1331]: geturl(flakyurl)
2020-09-07-14:48:19
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): snout:7777
DEBUG:urllib3.connectionpool:http://snout:7777 "POST /flakyservice HTTP/1.1" 406 27
2020-09-07-14:48:20
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): snout:7777
DEBUG:urllib3.connectionpool:http://snout:7777 "POST /flakyservice HTTP/1.1" 405 27
2020-09-07-14:48:21
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): snout:7777
DEBUG:urllib3.connectionpool:http://snout:7777 "POST /flakyservice HTTP/1.1" 405 27
---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
...

HTTPError: 405 Client Error: METHOD NOT ALLOWED for url: http://snout:7777/flakyservice

function 2 with try/except gets the correct errors but fails to invoke tenacity

In [1332]: @retry(reraise=True,wait=wait_fixed(1), stop=stop_after_attempt(3))
      ...: def geturl3(url):
      ...:         print(datetime.now().strftime('%Y-%m-%d-%H:%M:%S'))
      ...:         headers = {'user-agent': 'my-app/0.0.1','Content-Type':'application/json'}
      ...:         data = {'name':'testcall','service':'armor'}
      ...:         r=None
      ...:         try:
      ...:             r = requests.post(url, data=json.dumps(data), headers=headers)
      ...:             r.raise_for_status()
      ...:             return 'sent', r.status_code
      ...:         except requests.exceptions.HTTPError as err:
      ...:             return 'http failed', err
In [1334]: geturl3(flakyurl)
2020-09-07-15:23:00
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): snout:7777
DEBUG:urllib3.connectionpool:http://snout:7777 "POST /flakyservice HTTP/1.1" 404 27
Out[1334]:
('http failed',
 requests.exceptions.HTTPError('404 Client Error: NOT FOUND for url: http://snout:7777/flakyservice'))

EDIT: the fix is to r.raise_for_status() after the exception - this reraises and catches tenacity

1

There are 1 answers

0
js_cpnet On BEST ANSWER

Answered in Edited question: the fix is to r.raise_for_status() after the exception - this reraises and catches tenacity