Python retry package - tenacity : How to log the root cause of exception?

Question

As discussed in this question I am using tenacity to do retries.

A toy code looks like below

import logging
from tenacity import retry
import tenacity


@retry(wait=tenacity.wait_incrementing(start=10, increment=10, max=100), stop=tenacity.stop_after_attempt(3))
def print_msg():
    logging.info('Hello')
    logging.info("World")
    raise Exception('Test error')


if __name__ == '__main__':
    logging.basicConfig(
        format='%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s',
        datefmt='%d-%m-%Y:%H:%M:%S',
        level=logging.INFO)
    logging.info('Starting')
    print_msg()

The output looks like below

21-11-2018:12:40:48,586 INFO     [retrier.py:18] Starting
21-11-2018:12:40:48,586 INFO     [retrier.py:8] Hello
21-11-2018:12:40:48,586 INFO     [retrier.py:9] World
21-11-2018:12:40:58,592 INFO     [retrier.py:8] Hello
21-11-2018:12:40:58,592 INFO     [retrier.py:9] World
21-11-2018:12:41:18,596 INFO     [retrier.py:8] Hello
21-11-2018:12:41:18,596 INFO     [retrier.py:9] World
21-11-2018:12:41:18,596 ERROR    [retrier.py:22] Received Exception
Traceback (most recent call last):
  File "/Users/dmanna/PycharmProjects/demo/retrier.py", line 20, in <module>
    print_msg()
  File "/Users/dmanna/PycharmProjects/demo/venv/lib/python2.7/site-packages/tenacity/__init__.py", line 292, in wrapped_f
    return self.call(f, *args, **kw)
  File "/Users/dmanna/PycharmProjects/demo/venv/lib/python2.7/site-packages/tenacity/__init__.py", line 358, in call
    do = self.iter(retry_state=retry_state)
  File "/Users/dmanna/PycharmProjects/demo/venv/lib/python2.7/site-packages/tenacity/__init__.py", line 332, in iter
    six.raise_from(retry_exc, fut.exception())
  File "/Users/dmanna/PycharmProjects/demo/venv/lib/python2.7/site-packages/six.py", line 737, in raise_from
    raise value
RetryError: RetryError[<Future at 0x109fa6150 state=finished raised Exception>]

Can someone let me know how can I log the root cause of the exception?

Answer 1

you could use reraise=True option, there is no verbose doc, but you could read its source:

...
  File "53406953.py", line 10, in print_msg
    raise Exception('Test error')
Exception: Test error

Answer 2

you could see the whole exception chain under python3

exactly, which makes reraise=True unnecessary. also, with reraise=True you are in fact losing valuable information:

• reraise=True

$ python demo.py
09-03-2022:12:01:35,552 INFO     [demo.py:18] Starting
09-03-2022:12:01:35,553 INFO     [demo.py:8] Hello
09-03-2022:12:01:35,553 INFO     [demo.py:9] World
09-03-2022:12:01:36,553 INFO     [demo.py:8] Hello
09-03-2022:12:01:36,554 INFO     [demo.py:9] World
09-03-2022:12:01:38,556 INFO     [demo.py:8] Hello
09-03-2022:12:01:38,556 INFO     [demo.py:9] World
Traceback (most recent call last):
  File "demo.py", line 19, in <module>
    print_msg()
  File "/home/demo/.venv/lib/python3.8/site-packages/tenacity/__init__.py", line 324, in wrapped_f
    return self(f, *args, **kw)
  File "/home/demo/.venv/lib/python3.8/site-packages/tenacity/__init__.py", line 404, in __call__
    do = self.iter(retry_state=retry_state)
  File "/home/demo/.venv/lib/python3.8/site-packages/tenacity/__init__.py", line 360, in iter
    raise retry_exc.reraise()
  File "/home/demo/.venv/lib/python3.8/site-packages/tenacity/__init__.py", line 193, in reraise
    raise self.last_attempt.result()
  File "/home/user/.asdf/installs/python/3.8.12/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/home/user/.asdf/installs/python/3.8.12/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/home/demo/.venv/lib/python3.8/site-packages/tenacity/__init__.py", line 407, in __call__
    result = fn(*args, **kwargs)
  File "demo.py", line 10, in print_msg
    raise Exception('Test error')
Exception: Test error

• reraise=False

09-03-2022:12:02:07,444 INFO     [demo.py:18] Starting
09-03-2022:12:02:07,445 INFO     [demo.py:8] Hello
09-03-2022:12:02:07,445 INFO     [demo.py:9] World
09-03-2022:12:02:08,446 INFO     [demo.py:8] Hello
09-03-2022:12:02:08,446 INFO     [demo.py:9] World
09-03-2022:12:02:10,446 INFO     [demo.py:8] Hello
09-03-2022:12:02:10,446 INFO     [demo.py:9] World
Traceback (most recent call last):
  File "/home/demo/.venv/lib/python3.8/site-packages/tenacity/__init__.py", line 407, in __call__
    result = fn(*args, **kwargs)
  File "demo.py", line 10, in print_msg
    raise Exception('Test error')
Exception: Test error

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "demo.py", line 19, in <module>
    print_msg()
  File "/home/demo/.venv/lib/python3.8/site-packages/tenacity/__init__.py", line 324, in wrapped_f
    return self(f, *args, **kw)
  File "/home/demo/.venv/lib/python3.8/site-packages/tenacity/__init__.py", line 404, in __call__
    do = self.iter(retry_state=retry_state)
  File "/home/demo/.venv/lib/python3.8/site-packages/tenacity/__init__.py", line 361, in iter
    raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x7fbedb05d760 state=finished raised Exception>]