Async I/O Without Magic

Asynchronous Python lets one process make progress on other work while a coroutine waits for network or disk I/O. It does not make CPU work faster, make blocking libraries non-blocking, or remove resource limits. Correct async systems bound concurrency, set timeouts, propagate cancellation, and keep ownership visible.

What will you be able to do?

explain coroutines and the event loop;
run independent I/O concurrently with a limit;
use explicit timeouts and cancellation-safe cleanup;
choose sync or async based on dependencies and workload.

What happens at await?

An async def call creates a coroutine. When the running coroutine reaches an await on incomplete I/O, it yields control to the event loop. The loop can resume another ready task. If you call blocking code inside the loop, every coroutine waits.

python

import asyncio


async def classify(ticket_id: str, semaphore: asyncio.Semaphore) -> str:
    async with semaphore:
        await asyncio.sleep(0.05)  # stands in for async network I/O
        return f"{ticket_id}:classified"


async def main() -> None:
    limit = asyncio.Semaphore(3)
    async with asyncio.timeout(2):
        results = await asyncio.gather(
            *(classify(f"t_{number}", limit) for number in range(10))
        )
    print(results)


if __name__ == "__main__":
    asyncio.run(main())

The semaphore protects the downstream provider and your own sockets. The timeout bounds the whole operation. In real code, choose limits from measured capacity and provider quotas.

When is sync code better?

Use synchronous code when the dependency is synchronous, traffic is modest, or simplicity improves correctness. FastAPI can run sync route functions in a thread pool. Do not label a function async while calling a blocking database or HTTP client inside it; use a compatible async client or isolate blocking work deliberately.

CPU-heavy document parsing or embedding preprocessing belongs in a worker process or specialized service, not on the event loop.

Why does this matter in AI-native systems?

Model calls are high-latency I/O and may stream. Unbounded parallel calls can exhaust money, quotas, connections, and memory. Apply per-tenant quotas, global concurrency limits, timeouts, cancellation, and durable job state. Cancelling an HTTP response does not guarantee a provider stopped billing or a tool stopped executing.

Common mistakes

await inside a loop when calls are independent: consider bounded concurrency.
gather over thousands of jobs: use a worker queue and limited batches.
swallowing CancelledError: cleanup, then let cancellation propagate.
blocking SDK inside async def: event-loop stalls appear as system-wide latency.
assuming task completion is durable: process crashes erase in-memory tasks.

AI Pair-Programmer Prompt

text

Analyze this Python path for async correctness. Mark every I/O and CPU operation,
identify blocking calls on the event loop, define timeout and cancellation ownership,
calculate a safe initial concurrency limit from stated quotas, and propose a load
test. Do not replace durable background work with create_task.

Exercise

Modify the example so one ticket raises an error and another exceeds the timeout. Record which tasks complete, fail, or cancel; add cleanup logging in finally.

Acceptance criteria: the program never waits forever, concurrency never exceeds three, and failure behavior is explained from observed output.

Knowledge check

Does async speed up CPU-bound work?
Why bound model-call concurrency?
Is asyncio.create_task a durable queue?

Answers

No; use processes or specialized compute for CPU work.
To protect quotas, cost, connections, provider capacity, and latency.
No; a process restart loses the task and its state.

Completion checklist

I can identify blocking work.
My concurrent example has a limit and timeout.
I understand cancellation does not undo external side effects.

Primary references

Async I/O Without Magic

What happens at await?

python

import asyncio


async def classify(ticket_id: str, semaphore: asyncio.Semaphore) -> str:
    async with semaphore:
        await asyncio.sleep(0.05)  # stands in for async network I/O
        return f"{ticket_id}:classified"


async def main() -> None:
    limit = asyncio.Semaphore(3)
    async with asyncio.timeout(2):
        results = await asyncio.gather(
            *(classify(f"t_{number}", limit) for number in range(10))
        )
    print(results)


if __name__ == "__main__":
    asyncio.run(main())

The semaphore protects the downstream provider and your own sockets. The timeout bounds the whole operation. In real code, choose limits from measured capacity and provider quotas.

When is sync code better?

CPU-heavy document parsing or embedding preprocessing belongs in a worker process or specialized service, not on the event loop.

Why does this matter in AI-native systems?

Common mistakes

await inside a loop when calls are independent: consider bounded concurrency.

gather over thousands of jobs: use a worker queue and limited batches.

swallowing CancelledError: cleanup, then let cancellation propagate.

blocking SDK inside async def: event-loop stalls appear as system-wide latency.

assuming task completion is durable: process crashes erase in-memory tasks.

AI Pair-Programmer Prompt

text

Analyze this Python path for async correctness. Mark every I/O and CPU operation,
identify blocking calls on the event loop, define timeout and cancellation ownership,
calculate a safe initial concurrency limit from stated quotas, and propose a load
test. Do not replace durable background work with create_task.

Exercise

Modify the example so one ticket raises an error and another exceeds the timeout. Record which tasks complete, fail, or cancel; add cleanup logging in finally.

Acceptance criteria: the program never waits forever, concurrency never exceeds three, and failure behavior is explained from observed output.