Ask ten teams what their AI automation stack is and nine will name a model. That is the wrong answer. The model is maybe 20% of what makes a production agent reliable. The other 80% lives in orchestration, integrations, and observability, which nobody puts on a slide. After shipping AI systems across more than 1,000 projects, we have converged on a stack that is deliberately unglamorous. Here is exactly what we build with and, more importantly, why each piece is there.

The four layers of a real AI stack

Every production system we ship has the same four layers: a model layer that does the thinking, an orchestration layer that decides what happens when, a data and integration layer that connects to the real world, and an observability layer that tells us whether any of it is working. Skip a layer and the system feels impressive in a demo and falls apart in week three.

Most teams pour their energy into the top layer because it is the fun one. The reliability, the cost control, and the trust all come from the bottom two. We scope projects in the reverse order that people expect, starting with integration and monitoring.

Layer 1: The model layer

We are model-agnostic on purpose. No single provider is best at everything, and locking a whole system to one vendor means overpaying on easy tasks and underperforming on hard ones.

We route, we do not commit

A single workflow rarely uses a single model. A support agent might use a large frontier model for the one genuinely hard reasoning step, a small fast model for intent classification, and an embedding model for retrieval. Each step gets routed to the cheapest model that clears the quality bar for that step. This routing is where most of the cost savings in a well-built system come from, and it is invisible to the end user.

Open and closed, side by side

We mix open-weight and closed models in the same system. Closed frontier models still win on the hardest reasoning out of the box. Open-weight models win on cost, control, and deployment inside a client's own environment when data cannot leave the building. Picking a side is a marketing decision, not an engineering one. We pick per step.

Small models do more than people think

A surprising share of production work is classification, extraction, and routing, and small models handle these at a fraction of the cost and latency of frontier models. We reach for the smallest model that works and only escalate when the task genuinely needs more. It keeps systems fast and cheap without sacrificing the quality that matters. In practice, a well-routed workflow often runs most of its steps on small models and touches a frontier model only once or twice, which is where the real economics of a system are decided.

Layer 2: The orchestration layer

The orchestration layer is the logic that decides what the agent does, in what order, and when a human needs to step in. This is where a pile of model calls becomes a system.

We keep orchestration explicit rather than hoping a single mega-prompt will figure everything out. Each step has a defined input, a defined output, and clear success criteria. When a step can take an action in an outside system, we wrap it so we can see exactly what fired and whether it worked. For anything high-stakes, we design human-in-the-loop checkpoints directly into the flow so the agent proposes and a person approves.

The framework itself matters less than people assume. What matters is that the control flow is legible, testable, and replayable. A trendy agent framework with tangled logic is worse than a plain state machine you can actually reason about.

Layer 3: The data and integration layer

This is the hardest layer and the one that decides whether a project succeeds. A brilliant agent connected to nothing delivers nothing.

Retrieval over fine-tuning, most of the time

For giving an agent knowledge of a client's business, we default to retrieval rather than fine-tuning. Retrieval keeps the source of truth in a system you control, updates instantly when the underlying data changes, and makes it possible to cite where an answer came from. Fine-tuning has its place for tone and narrow formats, but for knowledge it is usually the slower, more brittle path.

Connecting to the systems you already run

Agents earn their keep by taking action in the tools a business already uses: the CRM, the ticketing system, the ERP, the data warehouse. We connect through APIs, webhooks, and increasingly the Model Context Protocol, which gives agents a standard way to reach external systems instead of a pile of one-off glue code. The integration work is routinely more than half the effort in an automation project, which is why we scope it before writing a line of agent logic.

The unglamorous plumbing

Queues, retries, idempotency, and rate limits are not exciting, but they are the difference between an agent that handles 10,000 tasks cleanly and one that double-charges a customer when an API times out. We treat this plumbing as first-class engineering, not an afterthought bolted on before launch.

Layer 4: The observability layer

AI agents fail quietly. They keep responding, keep calling tools, and keep producing confident output that happens to be wrong. Ordinary uptime monitoring will not catch it, so we build measurement into every system we ship.

Every agent action is logged as a structured event: the input, the model used, the tools called, the retrieved context, and the final output. Those events feed continuous evals that run against a held-out test set on a schedule, plus a thumbs up or down on every meaningful interaction. We track output quality and cost per task, not just whether the service is up. When quality drifts, we know in hours instead of months. If you want the deep version of this, we wrote a whole piece on how we monitor AI agents in production.

Why the stack is intentionally boring

The uncomfortable truth is that our specific tool choices are mostly interchangeable and change every few months as the space moves. The durable advantage is not the logos on the architecture diagram. It is the test sets, the evals, and the scoping discipline that wrap around whatever tools we use.

We would rather ship a plain stack with rigorous evaluation than a fashionable stack with none. A team that can measure quality will out-ship a team chasing the newest framework every time.

What this means for your build

If you are assembling an AI automation stack, three takeaways carry most of the weight. Route across models instead of committing to one, so you pay for capability only where you need it. Budget the majority of your effort for integration and data, because that is where projects actually stall. And build observability in from day one, not after the first quiet failure erodes trust.

That is the whole stack. No secret model, no magic framework. If you want a system built this way, that is exactly what our automation practice does, or you can tell us what you are trying to automate and we will scope it with you.