Every conference demo shows the same thing: an AI agent that books flights, writes code, and manages your calendar — all in one smooth take. The audience claps. The founder raises a Series B.

Then someone tries to deploy it in a real enterprise, and everything falls apart.

We've deployed AI agents inside Fortune 500 operations. Not demos. Not prototypes. Systems that run 24/7, make real decisions, and touch real money. Here's what we've learned about the gap between what looks good on stage and what survives production.

The Demo-to-Production Gap

In a demo, you control the inputs. The agent gets a well-formatted request, accesses a clean API, and returns a polished response. The happy path is the only path.

In production, inputs are messy. APIs time out. Data is stale. Users ask for things the agent wasn't designed for. Edge cases aren't edge cases — they're Tuesday.

The fundamental problem is that most agent architectures are optimized for capability, not reliability. They can do impressive things occasionally. Enterprises need systems that do predictable things consistently.

What Production Agents Need

After deploying Sentara's autonomous agents across call centers and operations teams, we've converged on five non-negotiable requirements:

1. Bounded Autonomy

The most dangerous agent is one that doesn't know its limits. Every production agent needs explicit boundaries:

  • Action scope — What can it do? What requires human approval?
  • Confidence thresholds — Below what confidence level does it escalate?
  • Blast radius limits — What's the maximum impact of a single decision?

We implement this as a permission system, not unlike Unix file permissions. Each agent has a capability matrix that defines exactly what it can read, write, and execute. No implicit permissions. No "it seemed reasonable."

2. Deterministic Fallbacks

When an LLM-powered agent encounters ambiguity, it needs a fallback that isn't "try again with a different prompt." Our agents use a tiered decision architecture:

  • Tier 1: Rule-based — If the situation matches known patterns, use deterministic logic. No LLM needed. Fast, predictable, auditable.
  • Tier 2: Model-assisted — If rules don't cover it, use the model with constrained output. Structured responses, validated against schemas.
  • Tier 3: Human escalation — If confidence is below threshold or the action exceeds scope, route to a human with full context.

The key insight: most agent interactions don't need AI at all. In our deployments, 60-70% of decisions are handled by Tier 1 rules. The LLM handles the remaining 30-40% where judgment is actually required.

3. Observable State

You cannot debug what you cannot observe. Every agent decision must produce a trace that answers three questions:

  • What did the agent perceive? (inputs, context, retrieved information)
  • What did the agent consider? (candidate actions, confidence scores, reasoning)
  • What did the agent do? (action taken, outcome, side effects)

We log every decision at every tier. When something goes wrong — and something always goes wrong — the investigation takes minutes, not days.

4. Graceful Degradation

Production agents must handle failure modes that demo agents never encounter:

  • API failures — The agent can't reach an external service. Does it retry? Queue? Fall back to cached data?
  • Model latency spikes — The LLM takes 30 seconds instead of 2. Does the user wait? Does the agent use a smaller model?
  • Context corruption — The conversation history is inconsistent. Does the agent hallucinate forward or reset?

Each failure mode needs an explicit strategy. "Retry three times then error" is not a strategy. It's giving up slowly.

5. Continuous Evaluation

Demo agents are tested once. Production agents are tested continuously. We run evaluation suites against our deployed agents every hour:

  • Synthetic scenarios that test known edge cases
  • Regression tests for previously encountered failures
  • Drift detection on decision distribution (if the agent suddenly starts escalating 3x more than usual, something changed)

The Architecture That Works

After multiple iterations, we've settled on an architecture we call Supervised Autonomy. The agent operates independently within defined bounds, but every session is scored by an async evaluation pipeline.

User Request → Router → [Rule Engine | Agent | Human Queue]
                              ↓
                    Action + Trace Log
                              ↓
                    Async Evaluation Pipeline
                              ↓
                    Score + Alert (if anomalous)

The evaluation pipeline doesn't block the agent. It runs after the fact, scoring decisions against quality criteria. If a decision looks wrong, it alerts the operations team. If a pattern of poor decisions emerges, it can automatically tighten the agent's confidence thresholds — effectively making it more conservative until a human reviews what's happening.

What Enterprises Get Wrong

Three patterns we see repeatedly:

"Let's start with a general-purpose agent." No. Start with the narrowest possible scope. An agent that handles one specific workflow brilliantly is infinitely more valuable than an agent that handles everything poorly. Expand scope after you've earned trust.

"The model will figure it out." The model is one component. The system around it — routing, fallbacks, evaluation, permissions — is what makes it production-ready. Spending 80% of your effort on prompt engineering and 20% on infrastructure is exactly backwards.

"We'll add guardrails later." Guardrails aren't a feature you bolt on. They're an architectural decision that shapes everything else. Adding safety to an unsafe system is rebuilding the system.

The Honest Truth

AI agents in production are less autonomous than you'd think and more useful than you'd expect. The value isn't in replacing human judgment. It's in handling the 70% of interactions that don't require judgment at all, and providing structured support for the 30% that do.

That's not as exciting as a conference demo. But it's what actually works.