How to catch custom exception from signal handler in asyncio?

Question

I'm having problems catching a custom exception when thrown from a signal handler callback when using asyncio.

If I throw ShutdownApp from within do_io() below, I am able to properly catch it in run_app(). However, when the exception is raised from handle_sig(), I can't seem to catch it.

Minimal, Reproducible Example Tested using Python 3.8.5:

import asyncio
from functools import partial
import os
import signal
from signal import Signals


class ShutdownApp(BaseException):
    pass


os.environ["PYTHONASYNCIODEBUG"] = "1"


class App:
    def __init__(self):
        self.loop = asyncio.get_event_loop()

    def _add_signal_handler(self, signal, handler):
        self.loop.add_signal_handler(signal, handler, signal)

    def setup_signals(self) -> None:
        self._add_signal_handler(signal.SIGINT, self.handle_sig)

    def handle_sig(self, signum):
        print(f"\npid: {os.getpid()}, Received signal: {Signals(signum).name}, raising error for exit")
        raise ShutdownApp("Exiting")

    async def do_io(self):
        print("io start. Press Ctrl+C now.")
        await asyncio.sleep(5)
        print("io end")

    def run_app(self):
        print("Starting Program")
        try:
            self.loop.run_until_complete(self.do_io())
        except ShutdownApp as e:
            print("ShutdownApp caught:", e)
            # TODO: do other shutdown related items
        except:
            print("Other error")
        finally:
            self.loop.close()


if __name__ == "__main__":
    my_app = App()
    my_app.setup_signals()
    my_app.run_app()
    print("Finished")

The output after pressing CTRL+C (for SIGINT) with asyncio debug mode:

(env_aiohttp) anav@anav-pc:~/Downloads/test$ python test_asyncio_signal.py 
Starting Program
io start. Press Ctrl+C now.
^C
pid: 20359, Received signal: SIGINT, raising error for exit
Exception in callback App.handle_sig(<Signals.SIGINT: 2>)
handle: <Handle App.handle_sig(<Signals.SIGINT: 2>) created at /home/anav/miniconda3/envs/env_aiohttp/lib/python3.8/asyncio/unix_events.py:99>
source_traceback: Object created at (most recent call last):
  File "test_asyncio_signal.py", line 50, in <module>
    my_app.setup_signals()
  File "test_asyncio_signal.py", line 25, in setup_signals
    self._add_signal_handler(signal.SIGINT, self.handle_sig)
  File "test_asyncio_signal.py", line 22, in _add_signal_handler
    self.loop.add_signal_handler(signal, handler, signal)
  File "/home/anav/miniconda3/envs/env_aiohttp/lib/python3.8/asyncio/unix_events.py", line 99, in add_signal_handler
    handle = events.Handle(callback, args, self, None)
Traceback (most recent call last):
  File "/home/anav/miniconda3/envs/env_aiohttp/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "test_asyncio_signal.py", line 31, in handle_sig
    raise ShutdownApp("Exiting")
ShutdownApp: Exiting
io end
Finished

Expected output:

Starting Program
io start. Press Ctrl+C now.
^C
pid: 20359, Received signal: SIGINT, raising error for exit
ShutdownApp caught: Exiting
io end
Finished

Is it possible to raise a custom exception from a signal handler in asyncio? If so, how do I properly catch/except it?

Answer 1

handle_sig is a callback, so it runs directly off the event loop and its exceptions are just reported to the user via a global hook. If you want the exception raised there to be caught elsewhere in the program, you need to use a future to transfer the exception from handle_sig to where you want it noticed.

To catch the exception at top-level, you probably want to introduce another method, let's call it async_main(), that waits for either self.do_io() or the previously-created future to complete:

    def __init__(self):
        self.loop = asyncio.get_event_loop()
        self.done_future = self.loop.create_future()

    async def async_main(self):
        # wait for do_io or done_future, whatever happens first
        io_task = asyncio.create_task(self.do_io())
        await asyncio.wait([self.done_future, io_task],
                           return_when=asyncio.FIRST_COMPLETED)
        if self.done_future.done():
            io_task.cancel()
            await self.done_future  # propagate the exception, if raised
        else:
            self.done_future.cancel()

To raise the exception from inside handle_sig, you just need to set the exception on the future object:

    def handle_sig(self, signum):
        print(f"\npid: {os.getpid()}, Received signal: {Signals(signum).name}, raising error for exit")
        self.done_future.set_exception(ShutdownApp("Exiting"))

Finally, you modify run_app to pass self.async_main() to run_until_complete, and you're all set:

$ python3 x.py
Starting Program
io start. Press Ctrl+C now.
^C
pid: 2069230, Received signal: SIGINT, raising error for exit
ShutdownApp caught: Exiting
Finished

In closing, note that reliably catching keyboard interrupts is a notoriously tricky undertaking and the above code might not cover all the corner cases.

Answer 2

If I throw ShutdownApp from within do_io() below, I am able to properly catch it in run_app(). However, when the exception is raised from handle_sig(), I can't seem to catch it.

Response to the query given above

Custom Exception Implementation:

class RecipeNotValidError(Exception):    
    def __init__(self):       
        self.message = "Your recipe is not valid"        
        try:            
            raise RecipeNotValidError
        except RecipeNotValidError as e:            
            print(e.message)

In the method handle_sig, add try and except block. Also, you can customize messages in custom exceptions.

 def handle_sig(self, signum):
        try:
            print(f"\npid: {os.getpid()}, Received signal: {Signals(signum).name}, raising 
                 error for exit")
            raise ShutdownApp("Exiting")
        except ShutdownApp as e:
            print(e.message)   

In response to your second query : Is it possible to raise a custom exception from a signal handler in asyncio? If so, how do I properly catch/except it?

In-built error handling in Asyncio. For more documentation visit https://docs.python.org/3/library/asyncio-exceptions.html

import asyncio
async def f():
    try:
        while True: await asyncio.sleep(0)
    except asyncio.CancelledError:  
        print('I was cancelled!')  
    else:
        return 111