Await Holding a Lock

Why you should never go to sleep while holding the only key.

The idea

To prevent race conditions, threads use Locks (Mutexes) to ensure only one thread modifies data at a time. This is fine if the modification is instant (e.g., changing a variable). But if a thread acquires a lock and then makes a slow network call (an HTTP request or database query), it goes to sleep (awaits) while still holding the lock. Every other thread is now completely blocked, waiting for the network call to finish. Your app grinds to a halt.

Step 1: Thread A wants to update a user's cache, so it grabs the Lock.

How it works (Release before IO)

The golden rule of concurrency is: Only hold locks around purely in-memory, CPU-bound operations. If you need to do I/O, do the I/O first, store the result in a local variable, and then grab the lock to update the shared state.

# THE BAD WAY (Awaiting while holding a lock)
async def update_user(user_id):
    async with lock:  # Thread acquires lock
        # Thread goes to sleep for 2 seconds waiting for DB.
        # ALL OTHER THREADS ARE BLOCKED!
        data = await db.fetch(user_id) 
        cache[user_id] = data
        
# THE GOOD WAY (Do I/O first, lock only for the update)
async def update_user(user_id):
    # Do the slow network call outside the lock
    data = await db.fetch(user_id) 
    
    # Grab the lock just for the microsecond it takes to update the dict
    async with lock:
        cache[user_id] = data

Cost

Structuring code this way takes more thought, and sometimes means fetching data that you might end up discarding if another thread beat you to it (optimistic concurrency). However, it prevents catastrophic thread-pool starvation.

Watch out for