StreamingResponse: Async Generator Chunks and Server-Sent Events (SSE)
- Last updated
- Prerequisites:
- Python Asynchronous Generators (`async def` with `yield`)
- HTTP Chunked Transfer Encoding
- fastapi
- streaming-response
- sse
- async-generator
- server-sent-events
- ai-streaming
Read at your depth
The practical view
`StreamingResponse(async_generator())` streams data chunk-by-chunk to the client using HTTP `Transfer-Encoding: chunked`. For Server-Sent Events (SSE), set `media_type='text/event-stream'` and yield strings formatted as `data: <content>\n\n`. It allows streaming large files, audio, and live LLM token completions without buffering the full response in server memory.
The same idea in other frameworks
`StreamingResponse(generator(), media_type='text/event-stream')`.NestJS `@Sse()` returning `Observable<MessageEvent>`.Spring WebFlux `Flux<ServerSentEvent<String>>`.`res.writeHead(200, {'Content-Type': 'text/event-stream'}); res.write(...)`.Legacy vs modern
Buffered Full String vs Async Generator StreamingResponse
Buffering LLM responses causes 10-second initial wait times; `StreamingResponse` yields tokens in real time.
full_text = await generate_all(); return {'text': full_text} // ❌ User waits 10s for first wordreturn StreamingResponse(token_stream(), media_type='text/event-stream') // ✅ NEW: Instant 0ms TTFBInterview gotchas
Context
Interviewers test your knowledge of client disconnect handling in streaming ASGI endpoints.
The mechanical answer
When the client disconnects, the next attempt to `send()` over the ASGI connection raises a `ClientDisconnected` exception (or the iterator detects cancellation). To prevent wasting backend resources (like OpenAI tokens or database bandwidth), check `await request.is_disconnected()` inside your generator loop, or catch `asyncio.CancelledError`.
Trap
If you do not check `request.is_disconnected()`, your generator will continue executing compute and consuming resources until the loop completes.
Context
Checks precision regarding synchronous vs asynchronous generators in streaming.
The mechanical answer
Yes, `StreamingResponse` can accept a synchronous generator (`def gen(): yield ...`), but Starlette runs it inside an iterator loop that can block the event loop if the generator executes synchronous I/O. Always use an **asynchronous generator (`async def gen(): yield ...`)** for non-blocking I/O streaming.
Trap
Using a synchronous generator that calls `time.sleep()` will block the entire FastAPI event loop during stream generation.