LLMOps in Practice: Shipping, Monitoring, and Rolling Back AI in Production
LLMOps is the set of routines for releasing, watching and reversing changes to an AI system that runs on a large language model. If your pilot just got its go decision, this is the lesson that keeps AI in production from quietly getting worse while nobody can say why, and from breaking on the day your vendor retires the model you pinned. You will leave with one document, the Production Release and Rollback Runbook, and you will have rolled the system back once for real before you ever need to.
The Problem: Three Changes, No Record, and What It Costs
The first alert is a message in a team chat channel, a little after noon: "is the bot broken?"
The service desk lead forwards it to the product owner of a customer assistant that has just moved out of pilot. Within an hour, with the engineer who built the pilot, they have pieced together what changed.
At 9:10 that morning a team member edited a prompt in the live configuration to fix one customer complaint. Over the weekend the vendor updated the model behind the name in your settings, and told nobody in particular. Overnight the retrieval index (the searchable copy of your documents the assistant looks things up in) had its weekly rebuild. Some answers are now better. Some are clearly worse.
Now answer the only question that matters: which of the three changes did it? You cannot. There is no previous version to go back to, because nobody saved one, and the only alert was a user asking whether the bot was broken.
That is the expensive problem of this lesson. It belongs to two people: the head of AI or platform engineering lead who inherits the pilot, and the business product owner who signed the go decision. Neither did anything wrong that morning. They are running a production system with pilot habits.
What happened to Knight Capital in 45 minutes?
Knight Capital lost more than $460 million in 45 minutes on 1 August 2012 because of a bad software release with no working controls around it. This is trading software, not AI, and I am using it on purpose: it is the cleanest release management failure on the public record, and every missing control has an exact twin in your AI system.
The facts come from the U.S. Securities and Exchange Commission's press release of 16 October 2013. Knight was a trading firm that handled customer share orders through an automated order router. The defect went back to 2005, when Knight moved a section of code in that router and left an old function defective. The function was not meant to be used, and Knight left it in the router.
In late July 2012, new code was incorrectly deployed to the same router. On the morning of 1 August, before the market opened, Knight's systems sent out 97 automated error emails. Nobody acted on them.
Then the market opened. Over the next 45 minutes, trying to fill 212 customer orders, the router sent more than 4 million orders into the market and traded over 397 million shares. The loss was more than $460 million, and the SEC later added a $12 million penalty.
The interesting part is not the bug. It is the SEC's list of what was missing:
- No reconciliation of orders leaving the router against orders that had actually been entered.
- No block that stopped trading when a capital threshold was crossed.
- Inadequate controls on code deployment and testing.
- No adequate written procedures for responding to an incident.
Read that list again as the owner of an AI system: no check that what goes out matches what came in, no spending limit, no controlled release, no written plan for a bad day, and alerts nobody owned. The code was the trigger. The missing controls set the size of the loss.
Three other ways the same gap shows up
You watch the model and miss the business. On 2 November 2021 Zillow announced it would wind down Zillow Offers, the business that used pricing models to buy and resell homes. The Zillow Group press release of that date reported an inventory write-down of about $304 million for the third quarter of 2021, a further $240 to $265 million of expected fourth-quarter losses, a planned workforce reduction of about 25%, and a Homes segment pre-tax loss of $421.6 million for the quarter. Chief executive Rich Barton put it in one sentence:
"We've determined the unpredictability in forecasting home prices far exceeds what we anticipated."
Secondary analysis published by KDnuggets in December 2021 describes the mechanics: models tuned to pay more to hit volume targets, price recommendations sometimes overridden by hand, and adverse selection (sellers accept offers that are too high and reject ones that are too low, so the deals you win are the ones you priced worst). It reports the business moving from profitable to roughly a 6% average loss per deal. Treat that as informed secondary reporting, not company disclosure. The lesson holds either way: a model can look acceptable on its own dashboard while the unit economics fail.
Your vendor's model moves under you. Chen, Zaharia and Zou tested the same named model at two points in time. In their arXiv paper of July 2023, GPT-4's accuracy on a task of telling prime numbers from composite numbers fell from 84% in March 2023 to 51% in June 2023. The customer changed nothing. The authors conclude that this shows "the need for continuous monitoring of LLMs." It is one task in one study, but it settles the principle: a hosted model is a dependency that can change without you shipping anything.
Nothing tells the system to stop. Mandiant's AI risk and resilience special report (Google Cloud, September 2026) describes an anonymised financial services firm whose ledger-reconciliation agent hit a corrupted value, looped, and made over 15,000 high-cost API calls in under an hour, running up about $50,000 in cloud charges; Lesson 14 on AI FinOps and controlling the AI bill takes that case apart.
Four stories, one shape. In none was "the model is bad" the main failure. The failure was missing operating controls around something that changes.
How LLMOps Works: The Bundle, the Gate and the Watch, the Limits and the Stop
The bakery with one signature loaf
Picture a bakery known for one signature loaf. Customers notice when it changes.
The loaf depends on the recipe card, the flour, the starter (the living culture in the fridge that gives the bread its character), the oven settings and the tools on the bench. Change any one and you get a different loaf, even if the other four are untouched.
Now add two awkward facts. The mill sometimes changes its blend without telling anyone: same bag, same label, different flour. And now and then it writes to say it will stop making your flour altogether, on a date of its choosing.
A careless baker edits the recipe card in pen, uses whatever flour arrives, and hears about problems from customers. A careful baker does five things:
- Keeps yesterday's recipe card, and keeps a sealed bag of the old flour.
- Bakes a test batch every morning and tastes it against a reference loaf before opening.
- Does that test batch every morning, even when nothing in the kitchen has changed, because the mill might have.
- Limits how much dough goes into the oven at once, so one mistake ruins a tray, not the day.
- Lets a named person pull a tray off the shelf if it tastes wrong, without calling the owner first.
That is LLMOps. The translation:
| In the bakery | In your AI system | Plain definition |
|---|---|---|
| Recipe card | Prompt template | The written instructions the model receives on every request. |
| Flour | Model | The large language model, usually hosted by a vendor. |
| Sealed bag of old flour | Pinned snapshot | A model ID that points to one dated version instead of "the latest". |
| The mill's discontinuation letter | Deprecation notice | The vendor's announcement that a model version will stop working on a set date. |
| Starter in the fridge | Retrieval index or data snapshot | The searchable copy of your documents the system looks up answers in. |
| Oven settings | Configuration | Settings such as output length, routing rules and thresholds. |
| Tools on the bench | Tool definitions | The actions the model is allowed to call, such as "move this email to a queue". |
| Morning test batch | Golden set | The fixed set of test cases with known good answers, built in Lesson 8. |
| Reference loaf | Baseline | The before-AI business measurement recorded in Lesson 10. |
| Pulling the tray | Rollback | Restoring the previous working version in one step. |
What is a release bundle?
A release bundle is everything that determines your system's behaviour, saved together under one version number. The unit you ship is not a prompt. It is five things: the model ID pinned to a dated snapshot, the prompt templates, the retrieval index or data snapshot, the tool definitions, and the configuration.
Most teams arrive at Level 3 with one of those under version control: the prompt, because Lesson 9 on versioned prompts taught them to treat prompts like code. That is the recipe card; the other four are still loose. On a morning like the one above, they can prove the prompt changed and nothing else.
Release bundle versioning fixes this with a simple rule. Any change to any of the five makes a new bundle version. Bundle v1.1 is v1.0 plus one recorded change. And the previous bundle stays stored, so that "roll back" means "restore v1.0, all of it", not "try to remember what the settings used to be".
Why does the golden set have to run twice?
The golden set runs twice because quality can drop two ways: you change something, or something changes under you. Each needs its own check.
The gate is the golden set run before every change, with a pass threshold. A change that scores below the threshold does not ship unless a named person signs an exception. It is the test batch tasted before opening, and Knight's missing "deployment and testing controls".
The watch is the same golden set run on a schedule against the live system, even in a week when you have shipped nothing. This is model drift monitoring in practice, and teams skip it because testing something you have not touched feels pointless. The 84% to 51% finding is why it is not: the mill can change the flour.
Alongside the watch you track two kinds of signal. Model signals tell you whether the system is behaving: golden-set pass rate, error or refusal rate, latency, tokens, cost. Business signals tell you whether the system is still worth running: the handling time, rework rate or other metric you baselined in Lesson 10. Zillow is the reason the second kind is not optional. This is deliberately a minimal LLM monitoring setup; Lesson 18 on AI observability adds tracing, continuous evaluation on live traffic and incident response.
What are the limits and the stop?
Limits are numbers the system cannot exceed, and the stop is a switch a named person may pull. Together they cap how bad a bad day gets.
- Maximum iterations: how many model calls one task may make before it hands over to a person.
- Token and spend budgets: a ceiling per task and per day.
- Circuit breaker: an automatic cut-off that trips when errors or spend cross a line, the way a fuse does.
- Kill switch: a manual off switch that falls back to the old process, held by a named person and deputy who need no permission to use it.
Knight had no capital-threshold block. The looping ledger agent in the Mandiant report had nothing that stopped the 15,001st call. Same gap, in a report published 14 years later.
What happens when the vendor retires your pinned model?
Your system stops working on the retirement date unless you have migrated, because a retired model no longer answers. Pinning protects you from silent change, not from scheduled removal: every pinned snapshot has an end date, announced or not.
Read your vendor's deprecation policy before you need it. As one example, Anthropic's model deprecations page promises "at least 60 days' notice before model retirement for publicly released models", defines a deprecated model as one that still works but has a recommended replacement and a retirement date, and says plainly that "requests to retired models fail". Its history shows the pattern: users of Claude Sonnet 4 and Claude Opus 4 were notified on 14 April 2026, and both models were retired on 15 June 2026. Other vendors publish different terms, so check yours and your contract.
The rule is short: a model retirement migration is a release like any other, except that the vendor chose the deadline. The replacement model makes a new bundle version and goes through the same gate, here called requalification: run the golden set against the replacement with your current prompts, compare with your recorded score, fix the prompts that regress, and rerun. Then release in stages and watch the business signals, because a newer model can pass your test cases and still change handling time or tone.
One detail catches teams out. Before the retirement date, your rollback target is the old bundle. After it, that bundle is dead, so the plan must name a second target: the previous prompts on the new model, or the manual queue the kill switch falls back to. Here is an illustrative schedule for a 60-day notice.
| Day | What happens | Who |
|---|---|---|
| 0 | Notice reaches a named owner; date written into the runbook; migration opened as a change | On-call owner |
| By 10 | Golden set run on the replacement with current prompts; score compared with the recorded one | Platform engineer |
| By 25 | Regressed prompts fixed, gate rerun, new bundle version frozen | Product engineer |
| By 35 | Staged release to a small share of traffic; business signals compared | Product owner |
| By 45 | Full release; rollback drill; post-retirement rollback target confirmed | Platform engineer |
| 46 to 60 | Buffer for the surprise you have not found yet | Nobody plans to use it |
This lesson handles one system's migration. When retirements arrive for many systems at once and you need fallback providers and contract terms, Lesson 25 on AI strategic resilience and model lock-in picks up the strategy.
Is this stack an industry standard?
No. The minimal stack here (bundle, gate, watch, two kinds of signal, limits, stop, owned alerts, override log, retirement watch, runbook) is a synthesis drawn from these failure cases, and you should not present it to your steering committee as a standard. The underlying debt has been named for a decade, though: Sculley and colleagues at Google named entanglement, configuration issues and changes in the external world among the risks in "Hidden Technical Debt in Machine Learning Systems" (NeurIPS 2015). Entanglement is the opening scene in one word: change anything and everything changes. It is also the short answer to LLMOps vs MLOps: same discipline, but you do not own the model, so it can drift or vanish without you shipping anything.
Where this sits on the ladder
This is the first lesson of Level 3, AI Operational, on the L1 to L5 ladder. Level 2 gave you the parts: a golden set (Lesson 8 on building a golden test set), versioned prompts (Lesson 9), and, in the previous lesson, AI pilot design with a baseline, success metrics and a signed go decision. Level 3 starts when those parts become a routine that runs without the pilot team. The next lesson, data architecture for AI with data contracts and semantic layers, uses the gate you build today to prove whether a contract or semantic layer actually improved answers.
The LLMOps Workshop: Build Your Release and Rollback Runbook This Week
This is a working session, not a design exercise. You are not choosing a platform; you are writing down how one system is released, watched and reversed, and then proving the reversal works. Teams often ask for an LLM deployment checklist; the runbook is that checklist, with proof attached.
Who is in the room, and what do you bring?
The method reads real settings and assigns real names, so the room is built around who can do that. Three people stay for the whole 2-hour session:
- The business product owner who signed the go decision, who alone can pick the business signal and accept a threshold.
- The engineer who built the pilot, who knows where every component actually lives.
- One platform or operations engineer with access to the live configuration, who will carry the pager afterwards.
Two more join for one part each. The service desk lead comes for steps 6 and 7, because the manual fallback and the alert route land on their team. Whoever receives the vendor's account emails, often procurement, comes for 10 minutes in step 2 to find where deprecation notices go today. Leave out architects and steering committee members; nothing here needs a decision above the product owner.
Bring four inputs: the pilot's go decision and baseline (Lesson 10), the golden set (Lesson 8), the versioned prompts (Lesson 9), and access to the live configuration, because you will read real settings off a real screen, not from memory. Book a separate 30-minute rollback drill later in the same week.
The nine steps
- List what changes behaviour. On a whiteboard, write every component that can alter an answer: model ID, prompt templates, index or data snapshot, tool definitions, configuration, and any manual override step where a person corrects the output. Next to each, write two things: who can change it today, and whether that change is recorded anywhere. "Anyone with admin access" and "no" are common answers, and they are the finding.
- Freeze a bundle and find its end date. Pin the model to a dated snapshot where your vendor offers one. Tag the prompt commit. Record the index build date. Give the set one bundle version, for example v1.0, and store the previous bundle where it can be restored; if there is none, today's becomes the first, and you say so in writing. Then look up the vendor's published retirement date for that snapshot, or write "none announced" with today's date, and route the vendor's deprecation notices to a named person. A retirement nobody hears about becomes an unplanned release.
- Wire the gate. Decide the pass threshold on the golden set that a change must meet before release, and name who can sign an exception. Then run the set against the current bundle right now, in the room, and record today's score. That number is your reference point for every future change, including the next model migration.
- Schedule the watch. Run the same golden set against the live system daily or weekly, shipped or not, and decide what drop from the recorded score triggers an alert.
- Choose model signals and business signals. Pick two model signals, for example golden-set pass rate and error or refusal rate. Pick two business signals straight from the Lesson 10 baseline, for example handling time and rework rate. Write the baseline value next to each. If the product owner cannot find the baseline, stop and find it; a signal with no baseline is decoration.
- Set limits and name the stop. Write the maximum model calls per task, the token and spend budget per day, and the names of the person and deputy who may switch the system off without asking. Say out loud what happens to the work when they do: which manual queue it falls back to.
- Route alerts to a person. Every alert goes to a named owner, with an acknowledgment time. No shared mailbox, no general channel. Remember the 97 emails.
- Log overrides. Decide where a human override of the system's output gets recorded, and who reviews that log weekly. Overrides feel like judgment, so nobody writes them down, yet they change what the system really does, and a rising count is often your earliest warning. The log will hold staff names and customer content, so give it the same retention rule as the system it comes from.
- Write the runbook and drill it. One page: symptoms, first checks, how to roll back, who to tell. Then, later in the week, perform a real rollback to the previous bundle in a test environment with a stopwatch running. Write down the time and everything that surprised you.
Step 9 is the one people negotiate away. A rollback you have never performed is a hope; the 30-minute drill nearly always finds a step nobody knew how to do.
One more of the LLMOps best practices to add once the basics work: release in stages. Send a new bundle to a small share of traffic, compare its signals with the old bundle, then widen it. The gate catches what your golden set covers; a staged release catches what it does not.
A filled-in example: the claims email assistant
This example is invented for illustration. Picture a 400-person insurance broker whose pilot assistant reads incoming claims emails, sorts them into queues and drafts a first reply for a handler to check. It passed its go decision against a baseline of 11 minutes average handling time per email.
The team runs the workshop. Step 1 produces the first surprise: three people can edit the live prompt, and the model setting names a model family, not a dated version. Step 2 produces the second: vendor notices go to the inbox of a contractor who set up the account during the pilot and has since left. They are rerouted to the platform engineer that afternoon. By the end of the two hours, the whiteboard looks like this.
| Bundle component | Current version | Who can change it | Gate before release | Rollback step |
|---|---|---|---|---|
| Model | Vendor snapshot dated in the ID, pinned; no retirement date announced as of the session | Platform engineer | Golden set, 120 emails, pass at 90% or above | Restore previous ID from bundle v1.0 |
| Prompt templates | Tagged commit, bundle v1.1 | Product engineer, by pull request | Same golden set | Revert to tag v1.0 |
| Retrieval index | Built Sunday night, policy documents | Data engineer | 20 retrieval spot checks | Repoint to last week's index |
| Tool definitions | Queue-routing tool, 1 tool | Platform engineer | Dry run on 10 emails | Disable tool, route to manual queue |
| Limits | 3 model calls per email, daily spend cap | Product owner | Not applicable | Kill switch held by service desk lead |
Model signals: weekly golden-set pass rate and the share of emails where the assistant returns an error. Business signals: average handling time against the 11-minute baseline, and the share of drafts a handler rewrites from scratch. Alerts go to the platform engineer on call, with a 30-minute acknowledgment time in business hours. Overrides (any email a handler moves to a different queue than the assistant chose) are counted from the queue system and reviewed by the product owner every Friday.
Then the drill, on Thursday. Rolling back the prompt takes 4 minutes. Rolling back the index does not happen at all: nobody present knows how to repoint it, and the one data engineer who does is on leave. That step gets a named owner and deputy in the runbook, and the drill is rerun the next week.
That discovery cost 30 minutes on a quiet Thursday. Picture finding it during an incident, or two weeks before a retirement date.
The artifact: your Production Release and Rollback Runbook
You finish with one runbook per AI system, two pages at most, which doubles as your AI rollback plan. Copy this table and fill in the right-hand column.
| Field | What to write |
|---|---|
| System name and business owner | The system, and the product owner who signed the go decision |
| On-call owner and deputy | Two named people, not a team name |
| Bundle version and components | Version number, plus the pinned version of the model, prompts, index or data snapshot, tools and configuration |
| Model retirement date | The vendor's retirement date for the pinned snapshot (or "none announced" and the date checked), who receives deprecation notices, and the migration owner |
| Evaluation gate | Which test set, the pass threshold, and who can approve an exception |
| Scheduled watch | How often the set runs against the live system, and the drop that triggers an alert |
| Model signals and business signals | Two of each, with the baseline value beside every one |
| Hard limits | Maximum iterations per task, token budget, spend budget |
| Kill-switch holder | Named person and deputy, and the manual fallback the work goes to |
| Alert routing | Named recipient and acknowledgment time |
| Override log | Where overrides are recorded, who reviews them weekly, and how long they are kept |
| Rollback steps | The steps in order, the last drill date, and the measured time in minutes |
| Incident contacts | Who must be told, in what order |
Leave an empty field visibly empty. An honest blank is a backlog item; a plausible guess is a future incident.
Failure Modes, How to Spot Them, and the Metrics to Track
Each failure mode can be found with one question. Ask them in your next operations review.
Seven failure modes and the question that exposes each
| Failure mode | What it looks like | How to spot it |
|---|---|---|
| Floating model alias | The configuration names a model family, not a dated snapshot, so the vendor decides when your behaviour changes. | Read the model ID in the live configuration. |
| Unowned retirement date | The snapshot has, or soon will have, a retirement date, and notices go to a shared or departed person's inbox. | Ask who received the vendor's last deprecation notice, and when. |
| Gate without a watch | Evaluations run only when the team ships something. | Ask for the last evaluation date in a week with no release. |
| Rollback on paper | A rollback procedure exists and has never been run. | Ask for the drill date and the measured time. |
| Alerts with no owner | Alerts land in a channel or shared mailbox. | Ask who acknowledged the last three. |
| Model metrics only | Dashboards show accuracy and latency, and nothing from the business baseline. | Look for the Lesson 10 metric on the dashboard. |
| Silent overrides | Staff correct or bypass outputs with no record. | Ask for the override count. "We don't know" is the finding. |
An eighth sits behind all seven: the pilot team becomes the production team by default. Nobody decided it, so there is no named on-call owner and no written incident procedure, and the seven follow naturally. If you fix only one thing this week, put two names in the on-call field.
Which LLMOps metrics should you track?
Track six metrics, each with a simple formula, and report them monthly to whoever owns the system's budget.
- Golden-set pass rate on the scheduled run. Weekly, against the score recorded in step 3. This is your drift detector.
- Share of production changes that passed the gate. Changes with a gate record, divided by all changes in the change log. Anything below 100% means someone is editing production directly.
- Time to roll back. Measured in the drill, in minutes. Not estimated.
- Alert acknowledgment time. From the alert timestamp to a named person's acknowledgment.
- Override rate. Logged overrides divided by total outputs, and its trend. A rising rate means the people closest to the work trust the system less than your dashboard does.
- Migration runway. Days between your replacement model passing requalification and the current model's retirement date. Negative, or untested, means you are already late.
Not on the list: number of prompts, users or benchmark scores. Those describe activity. These six describe control.
Frequently Asked Questions
How is LLMOps different from MLOps?
The discipline is the same, but with a vendor-hosted language model the model itself can change, or be retired, without you shipping anything. A July 2023 Stanford and Berkeley study (Chen, Zaharia and Zou) measured GPT-4 accuracy on one task falling from 84% to 51% in three months, which is why scheduled evaluation against the live model is required. There are also more parts to version: prompts, retrieval indexes and tool definitions.
Do we need to buy an LLMOps platform before we go to production?
No. For a first production system, a written runbook, a golden set on a schedule, a stored previous bundle and named owners give you the controls that mattered in every failure in this lesson. Tools can automate those routines later, but a tool with no named owner and no drilled rollback leaves Knight Capital's gaps open.
How often should we run evaluations on a model we have not changed?
Run them on a fixed schedule, daily or weekly, against the live system, with an alert threshold set as a drop from your recorded score. The point is to catch changes you did not make, so the schedule cannot follow your release calendar. Higher-volume or higher-risk systems justify daily.
What should an AI rollback plan include?
An AI rollback plan needs the whole previous bundle stored, the rollback steps in order with a named owner for each component, the manual fallback the kill switch sends work to, and a drill date with a measured time in minutes. During a model migration it also needs a second target for after the old model's retirement date, because the old bundle stops working then.
How much notice do vendors give before retiring a model?
It depends on the vendor, so read its published policy and your contract. Anthropic, for example, promises at least 60 days' notice before retiring publicly released models, and says requests to a retired model fail. Whatever the period, start requalifying the replacement against your golden set as soon as the notice arrives, not near the deadline.
Key Takeaways
- LLMOps is the routine for releasing, watching and reversing changes to an AI system. The model is rarely the main failure; the missing controls around it are.
- Ship a bundle, not a prompt: pinned model snapshot, prompt templates, retrieval index or data snapshot, tool definitions and configuration, under one version number, with the previous bundle stored.
- Run the golden set twice: as a gate before every change, and as a scheduled watch against the live model, because a vendor can change behaviour without you shipping anything (84% to 51% on one task, Chen, Zaharia and Zou, July 2023).
- A pinned model will be retired. Record its date, route deprecation notices to a named owner, and treat the migration as a release: requalify on the golden set, stage it, and name a rollback target for after the old model stops answering.
- Watch business signals from your pilot baseline next to model signals, set hard limits on iterations, tokens and spend, and name a person and a deputy who may pull the kill switch without asking.
- Send every alert to a named owner with an acknowledgment time, and log human overrides. Knight Capital's 97 error emails went unanswered before a loss of more than $460 million in 45 minutes (SEC, October 2013).
- Drill the rollback with a stopwatch. Your runbook is finished only when it carries a drill date and a measured time.
Skill.re