Python retry using tenacity without decorator

Question

I am trying to do a retry using tenacity (without decorator). My code looks like as explained here.

import logging
from tenacity import retry
import tenacity


def print_msg():
    try:
        logging.info('Hello')
        logging.info("World")
        raise Exception('Test error')
    except Exception as e:
        logging.error('caught error')
        raise e


if __name__ == '__main__':
    logging.basicConfig(
        format='%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s',
        datefmt='%d-%m-%Y:%H:%M:%S',
        level=logging.INFO)
    logging.info('Starting')
    try:
        r = tenacity.Retrying(
            tenacity.stop_after_attempt(2),
            tenacity.wait_incrementing(start=10, increment=100, max=1000),
            reraise=True
        )
        try:
            r.call(print_msg)
        except Exception:
            logging.error('Test error 2')
    except Exception:
        logging.error('Received Exception')

On executing the above code. The output looks like below with no retry

/Users/dmanna/PycharmProjects/demo/venv/bin/python /Users/dmanna/PycharmProjects/demo/retrier.py
25-11-2018:00:29:47,140 INFO     [retrier.py:21] Starting
25-11-2018:00:29:47,140 INFO     [retrier.py:8] Hello
25-11-2018:00:29:47,140 INFO     [retrier.py:9] World
25-11-2018:00:29:47,140 ERROR    [retrier.py:12] caught error
25-11-2018:00:29:47,141 ERROR    [retrier.py:31] Test error 2

Process finished with exit code 0

Can someone let me know what is going wrong as I am not seeing any retry in the above code?

Answer 1

This has been answered here. Cross posting the answer

Hum I don't think you're using the API correctly here:

r = tenacity.Retrying(
            tenacity.stop_after_attempt(2),
            tenacity.wait_incrementing(start=10, increment=100, max=1000),
            reraise=True
        )

This translates to:

r = tenacity.Retrying(
            sleep=tenacity.stop_after_attempt(2),
            stop=tenacity.wait_incrementing(start=10, increment=100, max=1000),
            reraise=True
        )

Which is not going to do what you want.

You want:

r = tenacity.Retrying(
            stop=tenacity.stop_after_attempt(2),
            wait=tenacity.wait_incrementing(start=10, increment=100, max=1000),
            reraise=True
        )

Answer 2

not using tenacity, and not without decorator, but hear me out: here's a hacky way to retry functions on the fly without having to decorate any given function definition with @retry. I had a problem where a couple packages were giving me issues and one of the classes was too convoluted to try to overwrite the class methods w/inheritance:

def retry(times=2, wait=1, method=''):
    def outer_wrapper(function):
        @functools.wraps(function)
        def inner_wrapper(*args, **kwargs):
            final_excep = None
            for i in range(times):
                final_excep = None
                try:
                    res = function(*args, **kwargs)
                    return res
                except Exception as e:
                    final_excep = e
                    print(f'error with {method}: {e}')
                    time.sleep(wait * (i/2))
                    pass

            if final_excep is not None:
                return False
        return inner_wrapper

    return outer_wrapper

def resilient_function(func, *args, **kwargs):
    return run_sheet_function(functools.partial(func, *args, **kwargs))

@retry(times=3, wait=3, method='run sheet function')
def run_sheet_function(func):
    return func()


##original function call
df = spread.sheet_to_df(index=0, start_row=2, formula_columns=['Monthly Service']) 
##new call
df = resilient_function(spread.sheet_to_df, index=0, start_row=2, formula_columns=['Monthly Service'])

Answer 3

Note for Python 3.4+ users (which supports annotations), to explicitly force a retry in Tenacity the general use case is to annotate it with a simple @retry and then raise the special Exception TryAgain.

@retry
def do_something():
    result = something_else()
    if result == 23:
       raise TryAgain

This approach similar to the answer for Python 2.7 on this page can also be implemented:

import tenacity

def do_something():
    result = something_else()
    if result == 23:
       raise TryAgain

r = tenacity.Retrying(
        stop=tenacity.stop_after_attempt(2),
        wait=tenacity.wait_incrementing(start=10, increment=100, max=1000)
    )
r.wraps(do_something)

For further detail, see the API documentation on whether to retry or the source that defines annotations.