A feature works in the demo, works in staging, gets shipped, and two weeks later support is quietly fielding complaints. This is such a common pattern that it is worth writing down the causes, because very few of them are about the model being bad at its job.
The evaluation set was the demo
The most common failure. Someone tries twenty inputs while building, they all look good, and those twenty become the de facto test suite.
Real traffic is not twenty inputs. It contains empty strings, forty page documents, mixed languages, copy and pasted spreadsheets, questions the feature was never meant to answer, and a surprising number of people testing whether they can make it say something rude.
The fix is boring: collect real inputs, keep the weird ones, and score against them before every release. If the evaluation set does not contain things that currently fail, it is not measuring anything.
Nobody is watching the output
Traditional monitoring watches for errors. A model that returns a fluent, confident, wrong answer returns HTTP 200 and no error at all.
Dashboards stay green while the feature quietly degrades. Unless you are sampling actual outputs and looking at them, or scoring them automatically, there is no signal. Silent quality decay is the characteristic production failure of this kind of system, and it is invisible to every alert you already have.
The prompt is code, but it was not treated as code
Prompts get edited in a hurry, sometimes directly in a dashboard, often without review, and rarely with a record of what changed.
Then behaviour shifts and nobody can say why. If a prompt lives outside version control, you have a production dependency with no history, no review and no rollback. That is not an AI problem. That is a configuration management problem that the industry solved decades ago and then forgot about because the new thing felt different.
The model changed underneath you
If you call a hosted model by a floating name rather than a pinned version, the thing on the other end can change without your involvement. Behaviour you tuned around can shift, formatting can drift, refusals can become more or less frequent.
Pin the version. Test before moving. Treat a model upgrade as a dependency upgrade, because that is exactly what it is.
Latency was measured as an average
Model latency has a long tail. An average of two seconds can hide a p99 of thirty, and the p99 is what people remember and complain about.
Worse, the retries meant to fix slow calls often multiply the load that caused the slowness. Timeouts, budgets and a fallback path matter more here than in almost any other kind of feature.
Cost was nobody’s job
A feature ships, usage grows, and the bill grows faster than the usage because each interaction turned out to involve several calls with a lot of context.
By the time somebody notices, the feature is load bearing and the options are all unpleasant. Track cost per interaction from the first day, not the first invoice.
The pattern
Look at that list again. Test coverage, monitoring, version control, dependency pinning, tail latency, and cost. Every one of them is a discipline that mature software teams already have.
AI features fail in production mostly because teams treat them as a new category of thing and quietly stop applying the practices they would never skip on a payments service. The novelty is real, but it sits in one narrow place, which is that the output is not deterministic. Everything else is the job you already know how to do.