Consider a very simple function:

def generate_something(data):
    if data is None:
        raise Exception('No data!')

    return MyObject(data)

Its output is basically an instance of an object I want to create or an exception if the function cannot create the object. We can say that the output is binary since it either succeeds (and gives back an object) or not (and gives back an Exception).

What is the most pythonic way to handle a third state, that is "success but with some warnings"?

def generate_something(data):
    warnings = []

    if data is None:
        raise Exception("No data!")

    if data.value_1 == 2:
        warnings.append('Hmm, value_1 is 2')    

    if data.value_2 == 1:
        warnings.append('Hmm, value_2 is 1')    

    return MyObject(data), warnings

Is returning a tuple the only way to handle this, or it is possible to broadcast or yield warnings from within the functions and catch them from the caller?

2 Answers

0
Netwave On

You can return a list of Exceptions (or custom exceptions) with the problems for later handling:

class MyWarning(Warning):
    pass

def generate_something(data):
    warnings = []
    if data is None:
        raise Exception("No data!")

    if data.value_1 == 2:
        warnings.append(MyWarnin('Hmm, value_1 is 2'))    

    if data.value_2 == 1:
        warnings.append(MyWarning('Hmm, value_2 is 1'))    

    return MyObject(data), warnings

And then for example:

def handle_warnings(warnings):
    for w in warnings:
        try:
            raise w
        except MyWarning:
            ...
        except AttributeError: #in case you want to handle other type of errors
            ...
3
Aran-Fey On

The built-on option: warnings

Python has a built-in warning mechanism implemented in the warnings module. The problem with this is that warnings maintains a global warnings filter, which might unintenionally cause the warnings your function throws to be suppressed. Here's a demonstration of the problem:

import warnings

def my_func():
    warnings.warn('warning!')

my_func()  # prints "warning!"

warnings.simplefilter("ignore")
my_func()  # prints nothing

If you want to use warnings regardless of this, you can use warnings.catch_warnings(record=True) to collect all thrown warnings in a list:

with warnings.catch_warnings(record=True) as warning_list:
    warnings.warn('warning 3')

print(warning_list)  # output: [<warnings.WarningMessage object at 0x7fd5f2f484e0>]

The self-made option

For the reason explained above, I recommend rolling your own warning mechanism instead. There are various ways to implement this:

  • Just return a list of warnings

    The easiest solution with the least overhead: Just return the warnings.

    def example_func():
        warnings = []
    
        if ...:
            warnings.append('warning!')
    
        return result, warnings
    
    result, warnings = example_func()
    for warning in warnings:
        ...  # handle warnings
    
  • Pass a warning handler to the function

    If you want to handle the warnings immediately when they're generated, you can rewrite your function to accept a warning handler as argument:

    def example_func(warning_handler=lambda w: None):
        if ...:
            warning_handler('warning!')
    
        return result
    
    
    def my_handler(w):
        print('warning', repr(w), 'was produced')
    
    result = example_func(my_handler)
    
  • contextvars (python 3.7+)

    With python 3.7 we got the contextvars module, which lets us implement a higher-level warning mechanism based on context managers:

    import contextlib
    import contextvars
    import warnings
    
    
    def default_handler(warning):
        warnings.warn(warning, stacklevel=3)
    
    _warning_handler = contextvars.ContextVar('warning_handler', default=default_handler)
    
    
    def warn(msg):
        _warning_handler.get()(msg)
    
    
    @contextlib.contextmanager
    def warning_handler(handler):
        token = _warning_handler.set(handler)
        yield
        _warning_handler.reset(token)
    

    Usage example:

    def my_warning_handler(w):
        print('warning', repr(w), 'was produced')
    
    with warning_handler(my_warning_handler):
        warn('some problem idk')  # prints "warning 'some problem idk' was produced"
    
    warn(Warning('another problem'))  # prints "Warning: another problem"
    

    Caveats: As of now, contextvars doesn't support generators. (Relevant PEP.) Things like the following example won't work correctly:

    def gen(x):
        with warning_handler(x):
            for _ in range(2):
                warn('warning!')
                yield
    
    g1 = gen(lambda w: print('handler 1'))
    g2 = gen(lambda w: print('handler 2'))
    
    next(g1)  # prints "handler 1"
    next(g2)  # prints "handler 2"
    next(g1)  # prints "handler 2"
    
  • without contextvars (for python <3.7)

    If you don't have contextvars, you can use this async-unsafe implementation instead:

    import contextlib
    import threading
    import warnings
    
    
    def default_handler(warning):
        warnings.warn(warning, stacklevel=3)
    
    _local_storage = threading.local()
    _local_storage.warning_handler = default_handler
    
    
    def _get_handler():
        try:
            return _local_storage.warning_handler
        except AttributeError:
            return default_handler
    
    
    def warn(msg):
        handler = _get_handler()
        handler(msg)
    
    
    @contextlib.contextmanager
    def warning_handler(handler):
        previous_handler = _get_handler()
        _local_storage.warning_handler = handler
    
        yield
    
        _local_storage.warning_handler = previous_handler