Gracefully exiting asyncio application

Continuing yesterday’s work on my simple supervisor library, I continued trying to propagate signals cleanly to child processes before exiting. I remembered that it isn’t enough to just propagate signals - you also have to actually reap them. This meant waiting for wait calls on them to return.

I had a task running concurrently that is waiting on these processes. So ‘all’ I had to do was make sure the application does not exit until these tasks are done. This turned out to be harder than I thought! After a bunch of reading, I recognized that what I needed to do was make sure I wait for all pending tasks before actually exiting the application.

This was more involved than I thought. It also must be done at the application level rather than in the library - you don’t want libraries doing sys.exit, and definitely don’t want them closing event loops.

After a bunch of looking and playing, it looks like what I want is in my application code is something like:

if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    try:
        loop.run_until_complete(main())
    finally:
        loop.run_until_complete(asyncio.gather(*asyncio.Task.all_tasks()))
        loop.close()

This waits for all tasks to complete before exiting, and seems to make sure all child processes are reaped. However, I have a few unresolved questions:

  1. What happens when one of the tasks is designed to run forever, and never exit? Should we cancel all tasks? Cancel tasks after a timeout? Cancelling tasks after a timeout seems most appropriate.
  2. If a task schedules more tasks, do those get run? Or are they abandoned? This seems important - can tasks keep adding more tasks in a loop?

I am getting a better handle on what people mean by ‘asyncio is more complicated than needed’. I’m going to find places to read up on asyncio internals - particularly how the list of pending tasks is maintained.

This series of blog posts and this EuroPython talk from Lynn Root helped a lot. So did Saúl Ibarra Corretgé (one of the asyncio core devs) talk on asyncio internals

Testing

Testing code that involves asyncio, signals and processes is hard. I attempted to do so with os.fork, but decided that is super-hard mode and I’d rather not play. Instead, I wrote Python code verbatim that is then spawned as a subprocess, and use stdout to communicate back to the parent process. The child process’ code itself is inline in the test file, which is terrible. I am going to move it to its own file.

I also added tests for multiple signal handlers. I’ve been writing a lot more tests in the last few months than I was before. I credit Tim Head for a lot of this. It definitely gives me a lot more confidence in my code.