"A production AI stack in Python: FastAPI and LangGraph patterns that hold up"
Python AI codebases have a characteristic way of rotting: dicts passed between functions, prompts inlined in loops, and a while-loop agent nobody can explain. The fix isn't more discipline, it's structure. Here is the FastAPI and LangGraph stack I run in production, with a public repo you can read.

There's a specific way Python AI codebases go bad, and if you've inherited one you'll recognise it immediately: dictionaries passed between functions with keys nobody documented, prompts built by string concatenation inside a while loop, and an "agent" that works most of the time for reasons nobody on the team can fully explain. Python's flexibility makes it the fastest language for AI experiments and, left unchecked, the fastest to produce an unmaintainable AI system.
The fix isn't heroic discipline. It's putting structure in the two places where structure pays: the boundaries of the system and the control flow of the agent. My stack for this is FastAPI at the edge and LangGraph in the middle, with Pydantic doing the enforcement throughout.
FastAPI earns its place because it makes the boundary contract-first by default. Every request body and response is a Pydantic model, validation happens before your handler runs, and the OpenAPI schema falls out for free. But the pattern that matters most is applying the same rule internally: model output gets parsed into a Pydantic model the moment it arrives, and nothing downstream ever touches the raw response. Pydantic's Rust-backed core makes that validation cheap enough to apply at every boundary without thinking twice. A ValidationError at the boundary is a good day, it means the bad data stopped where you could see it.

The same goes for state. The single biggest upgrade to any Python AI codebase is banning the anonymous dict. Every piece of state that moves through the system, the document being processed, the extraction attempts so far, the validation errors from the last try, lives in a typed model. When state is typed, your editor knows what exists, your tests can construct it precisely, and a refactor doesn't rely on grepping for string keys.
Then there's control flow, which is where LangGraph comes in, comfortably past its 1.0 release now, with a stable API and enough serious production users that the ground has stopped moving under it. The naive agent is a free loop: call the model, maybe call a tool, check if we're done, repeat. It demos well. In production it wanders, it retries forever, it revisits steps, it burns tokens on paths you never intended, and when it fails you have a transcript instead of a stack trace. A graph replaces that with explicit nodes and explicit edges: extraction happens here, validation happens there, and the only way to move between them is a transition you wrote down.
My repo at github.com/afareed007/docextract is the worked example, a document-extraction agent built exactly this way. There's a node that calls the model, a node that validates the output against Pydantic schemas with semantic checks layered on, and a conditional edge that routes failures back for a re-ask with the validation errors included in the prompt. The retry count lives in the typed state, so the loop is bounded by construction rather than by hope. When attempts run out, the graph routes to a terminal failure state instead of shipping unvalidated data.
Graphs also give you checkpointing, which matters more than it sounds. LangGraph can persist state after every node, which means a crash mid-run resumes from the last good step instead of starting over, and a flow can deliberately pause, say, at a human-review node, and pick up hours later when someone approves. Try retrofitting that onto a while loop.
Observability rounds it out. I trace runs with Langfuse so every node's prompt, response, latency and token count is inspectable, and I log structured events with the graph's current node name attached. "It failed in validate_output on attempt two" is a debuggable statement. "The agent got confused" is not.
I should be honest about the cost: this is more ceremony than a script. If your feature is one model call with one schema, you don't need a graph, a function with validation is fine, and reaching for LangGraph there is over-engineering. The structure pays off when there are multiple steps, retries, tool calls, or anything a human needs to approve; that's the point where free-form Python stops scaling and typed graphs start.
This is the work I do, building Python AI systems that are boring to operate, from the FastAPI boundary to the LangGraph core. If you've got an AI codebase that's drifting towards the unmaintainable version, get in touch via the contact form.
Senior engineering roles — or production AI consulting?
I'm open to senior AI and software engineering roles and to selective consulting on agents, RAG, and LLM automation in production. If this post resonates — as a hiring manager or as someone shipping similar work — get in touch.
Get in touch