You watched the pilot. It was good. The bot booked the appointment, read back the order, handled the awkward “actually, can you change that” without falling over. Someone on the steering committee said the word “transformational.” You signed.

Six months later it’s still in the pilot pen. Or it shipped, and now half the calls end with “let me put you through to someone who can help.” Same vendor. Same model. Same demo that wowed the room. So what changed?

Nothing changed. That’s the point. The pilot never touched the systems that production has to live inside — and the pilot was designed that way.

The number nobody puts on the slide

Here is the figure that should be on the first page of every business case and never is. In 2026, Gartner found that 64% of enterprise CX teams ran an agentic AI pilot — but only 27% had even one channel in full production. Most of the industry piloted. Roughly a quarter shipped.

That is not a model-quality problem. The models are extraordinary. They were extraordinary in your demo. The gap between 64 and 27 is almost entirely the work nobody demos: the wiring.

The model was never the hard part. The integration layer is where voice AI goes to die.

CX Today reported the same thing from the field this year — internally-built voice projects fail at roughly twice the rate of vendor-led ones, and the difference isn’t the AI. It’s the compounding cost of connecting that AI to the actual systems of record: the CRM, the order platform, the ticketing tool, the knowledge base, the four bits of middleware nobody has owned since 2019. The pilot skipped all of it. Production can’t.

Why the pilot is rigged to succeed

I don’t mean rigged dishonestly. I mean structurally. A pilot is built to prove the conversation works. So it’s given the cleanest possible conditions to do that.

The pilot reads from a spreadsheet of 200 happy-path records, not your live database with its duplicate accounts and half-migrated fields. It calls a mock API that returns in 40 milliseconds, not your order system that takes 1.8 seconds on a good day and times out on a bad one. It runs at five concurrent calls, not five hundred. It never meets the customer who’s been a member since 1996 and exists in three systems under two spellings.

Every one of those simplifications is reasonable for a proof of concept. Every one of them is a liability you’ve agreed to take on later without pricing it. The demo number isn’t your number — it’s the number you get when the hard parts are switched off.

The mechanism: latency and truth, not intelligence

Two things break in the move to production, and neither is “the AI got dumber.”

The first is latency, and it’s brutal because it’s invisible in the demo. A voice agent has to detect that the caller has interrupted, stop talking, and yield the floor in well under 200 milliseconds or the whole thing feels broken. Push total round-trip past 500ms and callers start talking over it. In the sandbox, your mock backend returns instantly, so this never shows. In production, every real lookup — balance, order status, eligibility — adds real time, and the agent either stalls mid-sentence or starts answering before it has the truth. The conversation that felt human in the demo now feels like a bad phone line.

The second is truth. A voice agent is only as good as what it can see. Wired to a mock, it always “knows” the answer. Wired to your real stack — with its outages, stale caches, and fields that mean different things in different systems — it now has to handle not knowing, gracefully, on a live call, without inventing an answer. That’s a design problem the pilot never posed. So the agent does the only safe thing it was taught: it escalates. Your containment collapses, and everyone blames the model.

What I’ve actually seen

The pattern is always the same, and it always arrives in week six, not week one.

The build that demos in a fortnight spends the next two quarters on integration — and it’s never the integration anyone flagged. It’s the CRM that returns a customer record in a slightly different shape for business accounts. It’s the telephony layer that drops DTMF tones under load. It’s the “real-time” inventory feed that’s actually a batch job from last night. It’s the auth flow that works for 95% of callers and locks out exactly the 5% who phone in angry.

None of that is exotic. All of it is the boring middle of every enterprise. And the teams that ship are not the ones with the best bot — they’re the ones who treated the conversation as the easy 20% and budgeted accordingly for the other 80%. The teams still stuck in the pen are the ones who saw a great demo and assumed the demo was the project.

The uncomfortable truth: a slightly worse model on a fully-wired, latency-tuned, failure-aware integration will beat a brilliant model on a fragile one every single time. Production rewards plumbing, not eloquence.

What to ask before you sign

Run every one of these before the business case, not after. The vendor who has good answers is the vendor who has shipped.

1. Show me the pilot’s data source. Live system of record, or a copied dataset? If it’s a copy, what’s different about the live one — duplicates, missing fields, business vs. consumer records?

2. What’s the real backend latency budget? Get the p95 round-trip time for every live lookup the agent makes, under production concurrency — not the demo’s instant mock.

3. What happens when a system is down or slow? Make them demo a timed-out CRM call live. Watch whether the agent stalls, hallucinates, or degrades gracefully.

4. How many concurrent calls has this exact configuration run? Pilot concurrency and production concurrency are different products. Ask for the number, not the brochure.

5. Who owns each integration in month seven? Name the human responsible for the CRM connector, the telephony layer, the knowledge sync — when something breaks at 2am.

6. Where does it escalate, and does that handoff carry context? A transfer that dumps the caller back to “how can I help you” is a failure wearing a success costume.

7. What’s the cost per resolved call at production volume — including the integration and maintenance you’re about to inherit — not cost per minute in the demo?

If a vendor flinches at any of these, you haven’t found a flaw in the questions. You’ve found the gap between 64 and 27.

The point

The pilot is the part everyone can do. Shipping is the part almost nobody does — and the difference is not the AI. It’s whether you treated the conversation as the project or as the trailer. Buy the wiring, not the demo. The bot that resolves the call is the boring one that’s plugged into everything.

A few quick questions I get asked

“Our pilot hit 80% containment. Why would production be lower?” Because the pilot answered questions it could always answer. Production includes the calls where the system of record is wrong, slow, or silent — and those are exactly the calls a clean pilot was built to avoid. Re-measure on live data before you promise the board a number.

“Isn’t this an argument for building in-house, so we control the integrations?” The opposite, usually. The field data this year shows internal builds failing at roughly twice the rate, precisely because the integration and operational scaffolding is harder and less glamorous than the model. Control the integrations through ownership and good contracts — not by underestimating how much of an org’s effort they’ll eat.

“How do I make the pilot more honest without blowing the budget?” One change: wire the pilot to one real system of record, at real latency, for one real journey. You’ll learn more from that single honest path than from ten happy-path demos. A narrow, true pilot beats a broad, fake one.

The Voice AI Brief is a weekly read on what actually works in enterprise voice AI — field notes from someone who’s been the buyer, the builder, and in the room. Subscribe to get the next edition by email.

If a vendor opens with their containment rate, be careful. It is the easiest number in this category to inflate, the easiest to misread, and the one most likely to make a deployment look like a success while your customers quietly suffer. After two years running enterprise voice AI in production at over a million minutes a month — and two decades in enterprise telephony before that — it is the metric I trust least when it’s said out loud in a demo, and the one I dig into hardest before I believe anything else.

Here’s the short version: containment rate tells you how many calls the AI handled without passing to a human. It does not tell you how many of those calls were actually resolved. Those are completely different things, and the gap between them is where voice AI projects go to die.

This matters whether you’re the one signing off on the contract or the one who’ll inherit the system afterwards and have to make the number real. The buyer gets sold the figure; the builder gets handed the consequences. Both are reading the same dashboard, and the dashboard is lying to both of them.

What containment actually measures — and what it hides

Containment is a deflection metric. A call is “contained” if it ended without reaching an agent. That sounds like success, and it gets presented that way: a high containment rate looks like cost saved, agents freed up, a system doing its job.

But a call can be contained for all the wrong reasons. The customer gave up. The system looped them until they hung up. It dumped them to a voicemail nobody checks. It confidently answered the wrong question and the customer rang back the next day — a second contained call that’s really one unresolved problem counted twice. Every one of those is a containment win on the dashboard and a customer-experience loss in reality.

A contained call and a resolved call are not the same call, and the distance between them is your customer’s experience.

That’s the whole essay in one line, but it’s worth sitting with, because almost every dashboard in this category is built to obscure it.

Of every 100 calls, 85 get “contained” — but only 57 are actually resolved. The 28 in between came back.

The number that actually matters

The metric that means something is resolution — how many contacts the AI genuinely closed out, such that the customer got what they needed and didn’t have to come back. Some teams call it autonomous resolution rate, some fold it into first-contact resolution. The label matters less than the discipline behind it: you are measuring outcomes, not avoidance.

Resolution is harder to game, because it forces the awkward follow-up questions:

• Of the calls we contained, how many customers called back within a day or two about the same issue? Repeat contacts are the truth serum for containment.

• What happened to CSAT on contained calls specifically, versus calls that reached an agent?

• When the AI handed off to a human, did it hand off early and cleanly, or only after the customer had already lost patience?

A system with 60% genuine resolution and clean handoffs is worth far more than one with 80% containment and a callback problem. The first is doing the job. The second is hiding the job.

Where I’ve seen this bite

The deployment I think about most reported a containment rate in the high 80s. On paper it was the best-performing bot in the estate, and it was held up in every internal review as the one to copy. Then we pulled the repeat-contact data and lined it up against it: roughly a third of the calls it “contained” came back within forty-eight hours — the same customer, the same unsolved problem, often a worse mood. Strip those callbacks out and the real one-and-done resolution was sitting closer to the high 50s. The headline number wasn’t lying about deflection; it was just silent about whether anything got fixed. CSAT on those contained calls was a full fifteen-odd points below the calls that reached a human, and nobody had been looking, because the dashboard everyone reported up the chain only had one number on it and that number was green.

The pattern, every time, is the same: the headline number is healthy, and the business underneath it is not. Nobody set out to mislead. Containment is just what the tooling measured by default, so containment is what everybody optimised — and optimising the wrong number is worse than not measuring at all, because it buys you false confidence right up until the volume hits.

Why the demo number is almost never your number

The containment figure in a sales deck was produced under conditions you will never replicate on day one:

• It was measured on a narrow set of well-behaved intents, not your full call mix.

• It assumes integrations that are already working — the CRM lookup, the order-status API, the authentication step. In production, those integrations are where things break, and a call the AI can’t complete because it can’t reach a backend system is a call it can’t contain.

• It rarely accounts for how your customers actually talk — accents, interruptions, background noise, the person who opens with three questions at once.

The honest way to read any vendor’s containment number is as a ceiling under ideal conditions, not a forecast. Your real number is a function of your call mix, your integration quality, and how much of your edge-case volume you’re willing to route to a human early. A good vendor will tell you that plainly — they’ll say you’ll start at a more modest number and climb as the system is tuned, and they’ll show you the curve from a real account. That honesty is a buying signal. A single, flawless, context-free figure is a selling one.

What to ask instead

If you’re evaluating voice AI right now, replace “what’s your containment rate?” with these:

• “Show me containment and resolution, side by side, on a call mix like mine.” If they can only show containment, you’ve learned something.

• “What’s the repeat-contact rate on contained calls?” The single most revealing question you can ask. Watch how quickly they answer it.

• “What happens when an integration is down or slow?” The honest deployments have a graceful answer. The fragile ones change the subject.

• “At what point does it hand to a human, and how do you decide that line?” Early, clean handoff is a feature, not a failure. A vendor who treats every handoff as a loss will build you a system that traps customers to protect its own metric.

• “What did this number look like in month one versus month six?” Real deployments climb. A flat, perfect number from day one is a demo, not a deployment.

The point

Containment rate isn’t useless — it’s just the wrong thing to lead with, and a dangerous thing to optimise for on its own. Optimise for containment and you can quietly build a system that’s brilliant at making calls end and terrible at making problems go away. Optimise for genuine resolution, with repeat-contact and CSAT as your guardrails, and containment takes care of itself as a by-product of actually helping people.

I’ve sat on every side of this — the buyer reading the deck and wondering if the number was real, the engineer finding out what the number cost at scale, and now the person in the room. The thing I’d tell my earlier self is the line from the top of this piece: a contained call and a resolved call are not the same call, and the distance between them is your customer’s experience. Measure that distance. Everything else is theatre.

A few quick questions I get asked

Is containment rate a bad metric? No — it’s a useful operational measure, but it’s the wrong thing to lead with or optimise in isolation. It tracks whether a call avoided a human, not whether the customer’s problem was solved. Pair it with a resolution measure and a repeat-contact rate.

What’s a good containment rate for enterprise voice AI? There’s no single right number, because it depends entirely on your call mix and integration depth. A realistic production figure on a sensible scope, climbing over the first few months, with low repeat contacts, beats a high headline number that hides callbacks.

What should I measure instead? Genuine resolution rate, repeat-contact rate on contained calls, and CSAT on contained calls versus agent-handled calls — reported weekly, AI and human side by side.

The Voice AI Brief is a fortnightly read on what actually works in enterprise voice AI — for the people who have to buy it, build it, and live with it. If a question about your own deployment is nagging at you, reply and tell me; reader questions shape what I write here. Subscribe to get the next edition by email.