"A production AI stack in TypeScript: what I actually use"

Your product runs on Node, your team writes TypeScript, and now you want to put AI features into it. The advice you'll hear is that AI means Python, so you start pricing up a second service, a second deployment pipeline, and a seam between two codebases that someone has to maintain forever. Before you do that, it's worth being clear about what production AI work actually involves, because for most product teams, it's work TypeScript is genuinely good at.

Calling a model is an HTTP request. The intelligence lives on the provider's side; your side is engineering. What you build around that request, validation, queueing, retries, logging, cost tracking, determines whether the feature survives contact with real users. None of that requires Python. All of it benefits from a type system, and TypeScript's is the best one sitting next to a mainstream product ecosystem.

So, the stack. For provider access I use the official SDKs, which are properly typed and well maintained. When I need to stream into a UI or keep a provider swap cheap, the Vercel AI SDK has earned its place, seven major versions in, it has grown from chat helpers into a serious agent toolkit, but I still keep abstractions thin, because every layer between you and the raw API is a layer that lags behind new provider features. Whichever client is underneath, it sits behind my own module with one exported function per capability, so the rest of the codebase never imports a vendor SDK directly. That module is the first file I walk a developer through when they join one of my projects, because everything else hangs off it.

Zod does more heavy lifting than anything else in the stack. Every boundary gets a schema: the request coming into the endpoint, the model output coming back, the job payload going onto the queue. Since Zod 4 the same schema converts to JSON Schema with a built-in call, so the contract you hand the provider's structured-output mode and the contract you validate the response against are one definition rather than two drifting copies. I use safeParse, never parse, because I want a result I can branch on rather than an exception escaping upwards. The payoff is a typed object that flows through the whole codebase with the compiler checking every touch point. LLM output is user input; Zod is where I enforce that.

Anything involving a model call goes through a queue, I use BullMQ on Redis. Model calls are slow, occasionally very slow, and they fail in ways your database never does: rate limits, timeouts, degraded responses. A queue gives you retries with backoff, concurrency limits that respect your provider quota, and a dead-letter path for jobs that will never succeed. It also forces you to make jobs idempotent, which you needed anyway the first time a retry double-sends an email.

Logging is structured or it's useless. I use pino, and every model call logs the same fields: a correlation ID, the prompt version, the model name, token counts in and out, latency, and whether validation passed. For proper traces across multi-step flows I add Langfuse, which gives you the full chain, prompt, response, retries, when you're debugging why one specific request went wrong at 2am three days ago.

Cost metering lives in the same client layer. Tokens are money, and "the bill doubled last month" is not a diagnosis. Because every call already reports token counts, aggregating spend per feature and per customer is a small addition, and it's the difference between guessing and knowing when someone asks which feature is burning the budget.

Where TypeScript genuinely shines is the absence of a seam. The schema that validates model output is the same schema that types your API response and your frontend props. One language, one deployment, one set of conventions, and your existing engineers can review the AI code because it's just their codebase. The async model fits too, LLM work is I/O-bound, which is exactly what Node was built for.

I'll be honest about the limits. If you're training models, doing serious data work, or want the deepest agent tooling, Python's ecosystem is ahead, new libraries land there first, and I use Python daily for exactly those jobs. But that's a reason to add Python where it earns its place, not a reason to move product-embedded AI features out of the stack your product already lives in.

This is the work I do, designing and building AI systems that hold up in production, in TypeScript and Python both. If you're adding AI to a Node product and want it built properly, get in touch via the contact form.