The question that separates vendors from operators is simple: who wakes up when the agent breaks at three in the morning? Not philosophically. Literally. Whose phone rings, what runbook do they open, and how long until the queue is healthy again?
Demos do not need an answer. Production does. An agent handling real claims, real dispatch decisions or real customer contacts is part of an operation now, and operations have always had an on-call discipline. The novelty of the technology does not exempt it.
The machinery nobody puts on a slide
A production agent ships with a named human on call, an escalation rota with real phone numbers, drift monitors that page before quality visibly degrades, and a documented rollback path that has been rehearsed at least once. None of this is exciting. All of it is the difference between a system and a liability.
An agent without an on-call rota is a demo that happens to be running in production.
The weekly evaluation report is the other half of the discipline. Every week, the agent retakes its exam: the same graded cases, plus new ones drawn from the live queue. The score goes to a person whose job includes reading it. Quality regressions get caught in days, not quarters.
Why this is the actual product
Models will keep improving and keep getting cheaper. The operating machinery around them is what enterprises actually buy: the guarantee that someone owns the result, measures it honestly and answers for it when it slips. That machinery is unglamorous, and it is the entire reason production AI works at all.