Asynchronous Programming with Python


Abstract

This tutorial intends to discuss about Python's approach to async IO. It aims to help you understand why we need it, and how could we use it to help with our development work.

Attention: asyncio evolves very fast since its introduction in Python 3.3.

Background

What if you could write a script that, instead of waiting for the second request, simply skip it and start executing the third request, then go back to the second one, and proceed from where it left off? That’s the nature of an asynchronous program. You minimize idle time by switching tasks.

Coroutines

Important: The term “coroutine” can be used for two closely related concepts:

  • a coroutine function: an async def function;

  • a coroutine object: an object returned by calling a coroutine function.

asyncio also supports legacy generator-based coroutines.

1. Legacy (Generator Based)

Note: Support for generator-based coroutines is deprecated since Python 3.8 and is scheduled for removal in Python 3.10.

Generator-based coroutines predate async/await syntax. They are Python generators that use yield from expressions to await on Futures and other coroutines. Generator-based coroutines should be decorated with @asyncio.coroutine, although this is not enforced.

2. Latest (Python3.7+)

Coroutines declared with the async/await syntax is the preferred way of writing asyncio applications.

Python Way to Async IO

Python3 now has a built-in implementation of Async IO: asyncio.

Further Investigations

1. Web Frameworks that support asynchronous APIs with async/await

  • Tornado: Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed. By using non-blocking network I/O, Tornado can scale to tens of thousands of open connections, making it ideal for long polling, WebSockets, and other applications that require a long-lived connection to each user.

  • FastAPI: FastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.6+ based on standard Python type hints.

  • Sanic: Sanic is a Python 3.6+ web server and web framework that’s written to go fast. It allows the usage of the async/await syntax added in Python 3.5, which makes your code non-blocking and speedy.

2. Alternative implementations of Asynchronous IO for Python

  • curio: Curio is a coroutine-based library for concurrent Python systems programming. It provides standard programming abstractions such as as tasks, sockets, files, locks, and queues. It works on Unix and Windows. You'll find it to be familiar, small, fast, and fun.

  • trio: A friendly Python library for async concurrency and I/O.

References

  1. Generators: The Final Frontier

  2. Async IO in Python: A Complete Walkthrough