How to Choose Your First AI Use Case and Avoid Pilot Purgatory
Picture the steering committee on a Monday morning (an illustration, but you have probably sat in it): a spreadsheet of 40 AI ideas on the screen, collected from every department, a third lifted from vendor demos, and a chair who has been asked to fund one before the quarter closes. The first AI use case you pick from that list decides more than one project. It decides whether the organization learns that AI can be measured, owned and finished, or learns that AI is a dozen pilots on a slide that nobody dares to stop. This lesson gives you a scoring sheet with written anchors, so two rooms score the same idea alike, a one-page charter with a measured baseline, and a dated decision, so that your first AI use case either ships or stops on purpose and never drifts into pilot purgatory.
The Problem: What the Wrong First AI Use Case Costs You
Start with a drive-thru lane.
In October 2021, McDonald's partnered with IBM to test an Automated Order Taker, a voice system that would listen to customers at the drive-thru speaker and enter their orders. The test grew to about 100 restaurants. A 2022 report by analysts at BTIG put order accuracy in the low-80% range, against a target of 95% or better before any wider rollout. In mid-June 2024 McDonald's confirmed it was ending the test "after thoughtful review", while saying it would keep working with IBM and would revisit voice ordering in future (Nation's Restaurant News, 17 June 2024). One caveat: the accuracy figure is an analyst's estimate, not a number McDonald's disclosed.
Most people tell this as a story about AI failing. It is more useful than that, with two halves.
What does the drive-thru story teach about selection?
It teaches that the task you pick sets the difficulty before anyone writes a line of configuration. The input was speech shouted over engine noise by strangers. The answer had to come back in real time. The customer heard the result directly, with no person checking it first. And the bar was very high, because an order that is wrong one time in five is a queue of annoyed people at the next window.
Customer-facing, real-time, messy input, a very high accuracy bar, and no human check. If you drew a map of every possible AI task, that is the hardest corner on it.
Why was ending the test a sign of good governance?
Because there was a number, and somebody acted on it. By the analysts' account, the test had a threshold for wider rollout, the reported result sat well below it, and the company closed the test and said so in public.
That is what a pilot is for. A pilot is a small, time-boxed trial that exists to produce a decision. A pilot that misses a threshold set in advance and is closed has done its job: you spent a limited amount to learn something true. Pilot purgatory is the opposite condition: a pilot with no success threshold and no end date, so it can never fail and never ship. It just continues, consuming two people's Thursdays and a licence fee.
How does a list of 40 ideas turn into pilot purgatory?
Through a meeting that never chooses. Nobody can compare 40 rows by eye, so the committee approves the three ideas whose sponsors spoke most confidently, tells two more teams to "explore", and asks for an update at the next meeting. Six months later those five are pilots. None has a baseline, meaning a measurement of how the work performs today. None has a date on which a named person decides to scale it or stop it. Then finance opens the budget cycle and asks what all this returned, and nobody can say.
At Level 1, the selection is usually made by whoever saw the latest demo or spoke last. Nobody chose badly on purpose. Nobody chose at all.
What do the numbers say about AI pilots that never reach production?
They say this is the normal outcome, though the surveys count different things, so read each one with its definition.
- S&P Global Market Intelligence (more than 1,000 respondents, reported by CIO Dive, 14 March 2025): 42% of companies abandoned most of their AI initiatives in 2025, up from 17% the year before, and on average 46% of AI proofs of concept were scrapped before production. The first figure counts companies; the second counts projects.
- Deloitte, State of AI in the Enterprise 2026 (21 January 2026, more than 3,000 executives): only 25% of respondents had moved 40% or more of their AI pilots into production.
- Gartner (press release, 29 July 2024) predicted that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025. That was a forecast whose outcome we have not seen verified, so do not quote it as a result.
- MIT NANDA, "The GenAI Divide" (covered by Fortune, 21 August 2025), reported that about 95% of generative AI pilots showed no measurable profit-and-loss impact. The figure is disputed: Fortune notes that NANDA, which builds agent infrastructure, may have an incentive to find current approaches wanting, though it saw no sign of skewed results; a published critique argues the success definition was so strict that short experiments counted as failures.
Now the balancing number. In a Gartner survey of 822 business leaders (September to November 2023, published in the same 29 July 2024 release), early adopters reported on average a 15.8% revenue increase, 15.2% cost savings and a 22.6% productivity improvement. The figures are self-reported, but the point is fair: value exists when a project lands. And a Gartner survey published on 30 June 2025 found that finding the right use case was a top barrier for 37% of low-maturity organizations. The rest of this lesson makes choosing a procedure.
How It Works: AI Use Case Prioritization With Value, Feasibility, Risk and Three Gates
Where you are on the ladder: this is Level 1 (AI Aware), lesson 3 of 5. Lesson 1 on shadow AI gave you the Shadow AI Register and the Data-to-Tool Rule. The previous lesson, the AI data inventory for mapping and classifying your data, rated the sources behind one candidate use green, amber or red. This lesson uses both to pick the single thing you will do first. The next lesson, building an AI literacy program that meets Article 4 of the EU AI Act, then trains the people your charter names before they touch the tool.
What is an AI use case?
An AI use case is one specific task, for one specific group, with one measurable result. "Use AI in customer service" is a wish. "Draft replies to shipment-status emails for the customer service team to check and send, cutting median handling time" is a use case. If you cannot say the task, the group and the number in one sentence, you do not have a use case yet.
How do you choose an AI use case that is safe to learn on?
You choose the car park, not the motorway. You learn to drive in an empty car park because a mistake there costs a dented cone, and you can make forty in an afternoon and learn from each. On the motorway your first mistake is also your last lesson. The drive-thru was the motorway. Your first case should be the car park: a place where the organization learns to steer, meaning to measure, review output, handle errors and make a decision, at low speed.
Reading the well-known failures in reverse gives you the profile of a good first case. This is our inference from those stories, not a finding from a study. A good first case is:
- Internal, or close to it. Staff see the raw output. If a customer ever reads the result, a person has approved it first.
- Tolerant of error. A weak draft costs minutes, not a refund or a headline.
- Human-reviewed. A named person checks every output before it goes anywhere that matters.
- Measurable in weeks. The task happens often enough that you have a real number inside a quarter.
That is what AI quick wins means when the phrase is used honestly: not the easiest idea on the list, but the one that produces a trustworthy number fastest at a low cost of error.
One more motorway story: Air Canada's website chatbot gave a customer wrong bereavement-fare advice and a tribunal held the airline to it; the Level 2 lesson on building a golden test set tells that case in full.
How do value, feasibility and risk fit together?
Choosing which room of a house to renovate first asks three questions: which room improves daily life most, which can we actually do, and which, if we get it wrong, floods the kitchen below? Gartner's version, set out by analyst Erick Brethenoux at its Data and Analytics Summit in Sydney in 2022 and reported by Computer Weekly, is to weigh business value against risk, then test feasibility: do we have the skills, the technology and the data to take this from inception through deployment and use? The aim is to separate the "no-brainers" from the cases that need caution. The same article quotes a practitioner who asks every business requester how they measure the existing process before any work begins.
This program turns that logic into an AI use case scoring matrix with three scores, each from 1 to 5. The sheet and its anchors are our construct, built on Gartner's reasoning, not a published Gartner standard, so do not present them as one.
- Value: hours or money a month, plus strategic fit.
- Feasibility: clean, available data, skills in house, and a tool to buy or switch on.
- Risk: when the output is wrong, who is harmed, whether a customer or regulator sees it, and whether a person checks it first.
You plot value against feasibility, the familiar value versus feasibility matrix. Risk works differently. It is a veto, meaning a score that removes an idea from the running no matter how good the other scores are. A case with Risk 4 or 5 cannot be your first case, however valuable. It waits in the backlog until the organization can drive. Why not average the three? Because averages hide exactly what you need to see. A chatbot with Value 5 and Risk 5 averages out to "promising". So did the drive-thru.
How do you score so that two teams get the same answer?
Write down what each number means before anyone scores, and score against the words, not against the other ideas. Without anchors, "Risk 4" means whatever the most senior person feels and the veto becomes a matter of mood. Adjust this starting set once, in writing, then leave it alone.
| Score | Value (hours a month, or the money equivalent) | Feasibility (data, skills, tool) | Risk (what happens when the output is wrong) |
|---|---|---|---|
| 1 | Under 20 hours, or no number anyone will sign | No tool and no skills in house, or a needed source is red or missing | Internal only, a person checks every output, no personal data |
| 2 | 20 to 80 hours | Needs a custom build or several integrations, or amber sources need major cleaning first | Internal only, personal data involved, a person checks every output |
| 3 | 80 to 200 hours | A tool exists, but contract checks pending or one integration needed, or a source is amber with no named fix | Output reaches customers or suppliers, but only after a person has checked each item |
| 4 | 200 to 500 hours, or a measurable effect on a named budget line | Tool to buy or switch on, contract checks passed, configuration only; sources green or amber with small named fixes, each with an owner and date | Output reaches customers or a regulator with spot checks only, or the activity is regulated (for example customs declarations) |
| 5 | More than 500 hours, or a direct effect on revenue or a contract penalty | As 4, and the tool is already licensed and the team has used it | Output reaches customers in real time with no person checking, or the use is high-risk under Annex III of the EU AI Act |
Three rules make the anchors hold.
- Size the hour bands once. The bands suit a firm of a few hundred people; a 5,000-person firm might multiply them by ten.
- Between two anchors, be conservative. Take the lower score for Value and Feasibility and the higher for Risk. Strategic fit can lift Value by one point at most, with the reason in writing.
- Annex III is a 5. Annex III of the EU AI Act lists high-risk uses such as AI for recruiting people or scoring individuals' creditworthiness; under the Digital Omnibus on AI (Regulation (EU) 2026/1744) their duties apply from 2 December 2027. Score any such use Risk 5 as a first case, whatever the human check, and confirm the list against the Official Journal text.
RAND's study of why AI projects fail (report RR-A2680-1, August 2024, 65 interviews) names root causes the three scores are built to catch: leaders misunderstanding the problem and how success will be measured (Value), insufficient quality data (Feasibility), and applying AI to problems beyond its capability (Risk).
What are the three gates every chosen case must pass?
Scoring picks the case; gates stop it sliding into purgatory. All three are written before work starts:
- A named owner. One person in the business, by name, who wants the result and will answer for it. A department is not an owner.
- A baseline for speed and for quality. How the task performs today, measured before any tool is switched on. If the threshold has a quality condition, that condition needs its own baseline, measured the same week.
- A kill date with a threshold. A calendar date on which a named person decides to scale or stop, and the minimum result that earns a "scale".
A restaurant trying a new dish does not reprint the menu. It runs a weekday lunch special: one dish, one named chef, a number on the kitchen whiteboard ("30 sold by Friday or it comes off") and a Friday decision. The whiteboard makes Friday easy: nobody argues and nobody is humiliated, because the rule was written on Monday. Stopping feels like failure only when nobody said in advance what stopping looks like. These gates are the Level 1 minimum; in Level 2, AI pilot design with baselines, success metrics and a go or no-go decision turns them into a full pilot charter and decision record.
Build vs buy AI: can switching on a vendor tool be your first use case?
Yes, often the most sensible one, if you score the task and not the licence. "Turn on the assistant in our office suite" is a purchase. "Use the suite's assistant to draft the weekly carrier performance summary for 12 account managers, who check it before it goes out" is a use case. Score the second sentence. Three things to watch:
- Value counts only the task you will measure. "Everyone saves time" is not a number, and the licence cost goes into the business case, not the score.
- Feasibility depends on two checks a demo never shows. First the contract: before a vendor route scores 4, IT security and legal confirm in writing that your inputs are not used for training, retention is stated, a data processing agreement is signed and data location is acceptable, and the indemnity position is recorded. Then permissions: an assistant built into software you already own works with whatever each user can already open. Microsoft's deployment guidance for its Copilot (Microsoft Learn, updated 28 October 2026) says to find and fix oversharing first: assess risky sites, run site access reviews, tighten sharing links and apply sensitivity labels. Until that is done for the task's sources, treat them as amber, and score Feasibility 3 at most until the clean-up has an owner and a date.
- Risk does not fall because you bought the tool. You answer for what it says, which is the point of the Air Canada ruling. Score Risk on where output goes and who checks it.
When an idea can be delivered by more than one route (switch on a feature you already license, buy a specialist tool, or build on a model provider's service), score each route as its own row. Value stays the same; Feasibility usually differs. The contested MIT NANDA study reported purchased tools succeeding about 67% of the time and internal builds about one-third as often (Fortune, 21 August 2025); treat that as cautiously as its headline. The direction matches common sense: you learn to measure and govern faster when you are not also learning to build software. If the scores put a build ahead, the scores win and the charter says why.
What does the AI business case for a first case need?
Three numbers and a name: what the task costs today, what the pilot costs to its kill date, the result that would justify scaling, and the person whose budget pays. That is the whole AI business case at Level 1. Pilot cost means licences, set-up hours and the owner's measuring time. The name is what committees skip: a steering committee without a budget line can recommend a pilot but not fund it, so the charter says whose cost centre pays to the kill date and who pays if it scales. Later, measuring AI ROI honestly across a portfolio (Lesson 19, Level 4) attributes value across many cases; at Level 1 you need one signed number and one budget holder.
The Workshop: Run the Scoring Matrix and Write the Charter in One Half-Day
You need one week of preparation and one three-hour session. You leave with a named artifact: the Use Case Scorecard and First Use Case Charter, with the anchor sheet attached so the next session scores in the same words.
Who needs to be in the room?
Each seat exists because one score or one check depends on it.
- The chair (transformation or operations lead): runs the clock, does not score.
- A finance partner: signs every Value score. No finance, no session.
- The IT lead, with the AI Data Inventory: settles Feasibility.
- Security or privacy: settles Risk.
- Procurement or the vendor manager, if any route is a purchase: brings the contract check status.
- Three to five function leads: own the ideas and the users.
- One frontline person per shortlisted idea, first hour only: corrects frequency and time per task.
Not in the room: vendors. Demos happen before or after, never during scoring. Bring the AI Data Inventory, the Shadow AI Register, and last year's list of process pain points if you have one.
What are the steps?
- Collect ideas as one-sentence use cases (the week before). Task, who does it, how often. Mine the Shadow AI Register first: it lists what staff already find useful enough to do without permission. Rewrite any idea that names a product as the task the product would do, and give each delivery route its own row.
- Screen out the non-starters (the week before, 30 minutes). The IT lead and privacy remove anything that fails the Data-to-Tool Rule or depends on a source rated red. Circulate the anchor sheet two days ahead.
- Calibrate on one idea (session, 20 minutes). Everyone scores one idea nobody is attached to, alone and on paper, then all reveal at once. Wherever two people are two or more points apart, read the anchor aloud, agree what it means and write the clarification onto the anchor sheet. This is the step that makes scores comparable.
- Score Value with finance (40 minutes). Estimate hours a month at the minimum acceptable result, not at the hoped-for target, and place each idea in a band. Finance signs. A value column that says "high" where it should say "215 hours a month" was scored by enthusiasm.
- Score Feasibility and Risk, silently first (50 minutes). Each scorer writes both numbers before anyone speaks; the IT lead settles Feasibility and security or privacy settles Risk. Gaps of two points go back to the anchor wording, not to a vote; nobody scores their own idea.
- Plot, veto and pick one (30 minutes). Place each idea on a value versus feasibility whiteboard, strike through anything with Risk 4 or 5, and from the high-value, high-feasibility corner pick the one with the lowest risk. One, not three: three first cases with the same two people is how purgatory starts.
- Draft the charter (30 minutes). Owner, decision maker, budget holder, how both baselines will be measured, target, minimum acceptable result and kill date. In the week after the session, before switch-on, the owner measures both baselines; the charter is not signed until both numbers are in.
- Park the rest and publish the backlog (10 minutes, then one email). Every other idea goes into a dated backlog with its scores and what would have to be true to revisit it. Within two working days the chair tells every proposer where their idea ranked.
What does a filled-in scorecard look like?
This example is illustrative, not a real company. Picture the same 420-person freight forwarder from Lessons 1 and 2. Its list started at 40 ideas and 14 survived the screen. Here are six rows, including one idea scored on two routes.
| Idea | Route | Value | Feasibility | Risk | Decision |
|---|---|---|---|---|---|
| Draft shipment-status replies; agent edits and sends | Add-on module of the service platform already licensed | 4 | 4 | 3 | Chosen |
| Draft shipment-status replies; agent edits and sends | Build on a model provider's service | 4 | 2 | 3 | Same task, slower route |
| Customer-facing tracking chatbot | Specialist vendor | 4 | 2 | 5 | Vetoed as a first case |
| Automatic customs classification | Build | 5 | 2 | 4 | Vetoed; regulated, parked |
| Summarize carrier contracts for procurement | Office suite assistant | 3 | 3 | 2 | Backlog, second |
| Meeting recaps for managers | Built-in feature of the meeting tool already licensed | 2 | 5 | 2 | Backlog; low value |
The tracking chatbot, the sales director's favourite, answers customers in real time with no person checking: Risk 5 on the anchors, no debate needed. Customs classification had the highest value in the room, but it is regulated and its data is thin. This is where the veto earns its keep: without it, the highest-value idea wins and the firm starts on the motorway.
Carrier contract summaries would read a procurement share that half the company can open, so its data counts as amber, and the clean-up of the sharing links has no owner or date yet: Feasibility 3. Meeting recaps is the easiest item, but finance could not attach meaningful hours to it. Easy and low-value is its own trap: you would finish it and learn nothing.
Shipment-status replies wins without being anyone's favourite. Be honest about its risk: the customer reads the reply, so it is Risk 3, not internal; what keeps it off the motorway is that an agent checks every draft before it leaves. The task happens about 1,000 times a week, and at the minimum acceptable 8 minutes against today's 11 it frees about 215 hours a month: Value 4, signed by finance. The add-on module had passed its contract checks, and its three sources are amber with small named fixes from the inventory (timestamps for two carriers, de-duplicating the mailbox, flagging stale contacts), with the red rate cards excluded, so Feasibility is 4. The build route loses on Feasibility, not on taste.
The room drafts the charter, and the owner fills in both baselines the following week:
- Use case: draft replies to shipment-status emails in the service platform; agents check, edit and send every one.
- Owner: the customer service lead. Decision maker and budget holder: the operations director.
- Users and route: 25 day-shift agents; buy (add-on module), contract checks confirmed in writing.
- Speed baseline: 11 minutes median handling time per status email, measured over one week before switch-on.
- Quality baseline: 3.1% of status replies trigger a customer re-contact within 48 hours, measured over the same week.
- Pilot cost: 25 seats of the add-on to the kill date plus IT and security set-up hours, charged to the operations budget.
- Target: 6 minutes.
- Minimum acceptable result: 8 minutes or less, with a re-contact rate no higher than 3.1%.
- Kill date: a Friday 10 weeks after switch-on, entered in the operations director's calendar as a date.
The minimum acceptable line has two parts on purpose, and each has its own baseline. Speed alone is easy to fake: people send the draft unread. The quality number must exist before the tool does. A "correction rate" fails that test, because before the pilot there are no AI drafts to correct; a re-contact (the same customer writing again about the same shipment within 48 hours) is already logged by the service platform. On the kill date the operations director looks at two figures. At 7 minutes and 2.9%, it scales. At 7 minutes and 4.0%, it does not, however fast it is. At 10 minutes it stops, the lessons are written down and the contract summaries move up. Each outcome is a success for the method.
What goes in the AI use case template?
The Scorecard uses the columns of the worked example, one row per idea and route, plus a one-line basis for each score signed by whoever settled it (finance for Value, IT for Feasibility, security or privacy for Risk). The Charter is one page for the chosen case:
| Charter field | What to write |
|---|---|
| Use case sentence | Task, group, measurable result |
| Owner, decision maker, budget holder | Names, not departments |
| Users and in-scope data sources | Who uses the tool; sources from the inventory with their ratings |
| Route and contract checks | Switch on, buy or build, and why; for vendors, the written confirmation |
| Speed baseline and date measured | Today's number, and the week it was taken |
| Quality baseline and date measured | A quality number that exists before the tool does, measured the same week |
| Target and minimum acceptable result | The hoped-for result; the "scale" threshold with a speed and a quality condition |
| Human review step | Who checks output, and at what point |
| Pilot cost and funding | Licences, set-up hours, whose cost centre pays to the kill date |
| Kill date and next in backlog | A calendar date, not a duration; the idea that moves up next |
If the charter does not fit on one page, the use case is not one use case.
Failure Modes, How to Spot Them, and the Metrics to Track
Each has a tell you can check in ten minutes with the deck and the charters.
| Failure mode | How to spot it |
|---|---|
| Pilot purgatory | A pilot older than its planned length with no decision recorded; no threshold in writing |
| Demo-driven choice | The chosen case appears on no scorecard, or was scored after it was chosen |
| Scores drift between rooms | Two groups score the same idea two or more points apart; no anchor sheet is attached |
| Licence mistaken for a use case | The use case sentence names a product, not a task and a group |
| Motorway first | The first case is customer-facing, real-time and has no human check |
| Quality condition with no baseline | The threshold says "no rise in errors" but nobody measured errors before the start |
| Value scored by enthusiasm | Finance was not in the room; the value column has no hours or money |
What do you do with the pilots already running?
Run them through the three gates retroactively. For each pilot, ask for the owner's name, the speed and quality baselines, and the kill date. Anything that cannot produce them gets two weeks to write a charter, or it is closed and moved to the backlog with a score. A short list where every row has a name, a number and a date beats a column of "in progress" in any budget conversation.
Which metrics tell you the method is working?
- Share of active pilots with an owner, both baselines and a kill date. Your headline number.
- Number of pilots past their kill date without a decision. The target is zero.
- Weeks from idea to scored decision.
- Share of pilots that reach a scale-or-stop decision on time. A "stop" counts as a success here.
- Calibration spread. How far apart scores were in step 3. It should narrow each session.
- The chosen case's two numbers, tracked weekly. In the example, median handling time and the 48-hour re-contact rate.
Report the first two at every steering meeting: they separate a portfolio from a slide.
Frequently Asked Questions
What is a good AI quick win for a first project?
A good AI quick win is a frequent internal drafting, summarizing or searching task where a person checks every output before it leaves, such as drafting replies to routine status emails for an agent to edit and send. It should produce a trustworthy speed and quality number within a quarter.
Can an AI feature in software we already license count as a first use case?
Yes, provided you score a specific task it will do, not the licence. Before it scores well on feasibility, confirm in writing that inputs are not used for training, retention is stated, a data processing agreement is signed and data location is acceptable, record the indemnity position, and fix overshared permissions on the sources it will read.
How do you get finance to fund a first AI use case?
Bring finance into the scoring and let them sign the value number, then show three figures: what the task costs today in hours, what the pilot costs to its kill date, and the minimum result that would justify scaling. Name the budget holder in the charter, because a steering committee without a budget line can only recommend a pilot.
Who should own the first AI use case?
One named person in the business who wants the result and will answer for it, usually the manager of the team that does the task. A department, IT or a vendor cannot own it; a separate named decision maker decides on the kill date.
What percentage of AI pilots fail?
On the best-documented count, about half: S&P Global Market Intelligence (reported March 2025) found an average of 46% of AI proofs of concept scrapped before production. Higher figures, such as MIT NANDA's 95% (August 2025), use stricter definitions and are disputed, so quote any such number with its publisher, date and definition.
Key Takeaways
- An AI use case is one task, for one group, with one measurable result; if it names a product instead of a task, it is not ready to score.
- Score every idea and every delivery route from 1 to 5 on Value, Feasibility and Risk against written anchors, calibrate on one idea first, and treat Risk 4 or 5 as a veto for a first case.
- Start in the car park: internal or human-checked before anything leaves, tolerant of error and measurable in weeks.
- Buying or switching on a vendor tool is a legitimate first case once the contract checks and the permission clean-up are done; buying does not lower the risk score.
- Every chosen case passes three gates before work starts: a named owner, speed and quality baselines measured before switch-on, and a dated kill date with a threshold.
- A pilot closed against a pre-set threshold is governance working; pilot purgatory is the pilot with no threshold and no end date.
- Your artifact is the Use Case Scorecard and First Use Case Charter with its anchor sheet. The next lesson trains the users it names.
Skill.re