The quarterly program review is forty minutes old and the chief financial officer (CFO) has asked one question twice. The first time, the answer was easy: the slide showed cumulative annualized savings, the line went up and to the right, and the room relaxed. The second time she asked it differently. "Fine. Now tell me whether it will still be true in two quarters." Nobody speaks, because the page cannot speak. It has one number on it. A number can say what happened; it cannot say what kind of shape the program is in while it happens. Somewhere in the building, gate adherence has been quietly falling for nine weeks under a throughput push, and the savings figure on the screen is rising partly because of it. Nothing on this page is a lie. The page is simply built so that the most important thing about the program is invisible on it.

The Two Ways a Program Dies, and Why They Are Opposites

You have spent Level 4 building the machinery: a portfolio sequenced across waves, foundations underneath it (the data program, the governance catalog, the platform records, the vendor register, the boundary map), and a funded 70 percent workstream (change plan, training blueprint, incentive audit, champion network, role briefs). BCG's 10-20-70 rule holds that roughly 10 percent of AI value comes from algorithms, 20 percent from technology and data, and 70 percent from people and process, and you have paid for all three parts. Now the program has to prove itself, quarter after quarter, to people who did not build it and will not read your workbook.

In Level 3 you learned to measure a single project honestly: a baseline, a like-for-like window, a delta with its controls named, a verification tax counted so the number was net rather than gross, and a confidence word attached. That instrument, the Delta Table, answers "did this pilot do anything?" It was never designed to answer the question a program faces: is the whole thing working, and is it working in a way that will keep working?

The temptation at this point is to solve the problem with arithmetic: add up the deltas, divide by the spend, report one return-on-investment (ROI) figure per quarter. Resist it, because a single number is not merely incomplete here. It is actively worse than useless, and the reason is that programs die of two diseases that are opposites, and a single value number is blind to both.

Disease one: stalled value

Stalled value is activity everywhere and nothing in the profit-and-loss statement (P&L). Tools deployed, licenses renewed, dashboards populated, town halls held, and at the end of the year the finance team cannot find the money. This is not a hypothetical failure mode; it is the base rate. McKinsey's State of AI work found 88 percent of organizations using AI regularly, and only about 39 percent able to attribute any impact to earnings before interest and taxes (EBIT), with most of those reporting under 5 percent. That gap between 88 and 39 is stalled value measured across an entire economy. MIT's finding that 95 percent of enterprise generative AI pilots produce no measurable P&L return is the same disease diagnosed at project level, and Gartner's forecast that over 40 percent of agentic AI projects will be canceled by the end of 2027 names unclear business value as a leading cause of death. S&P Global found 42 percent of companies scrapped most of their AI initiatives in 2025, up from 17 percent the year before.

A value-only scorecard is supposed to catch stalled value, and it half does. What it cannot do is catch it early, because value is a lagging indicator. By the time the money is visibly absent, you have spent four quarters producing the absence.

Disease two: hidden risk

The second disease is the opposite shape and the one that ends careers rather than budgets. Hidden risk is real value being generated on an eroding control base. The numbers are true. The savings are banked. And underneath them, gates are being rushed, override rates are climbing, escapes are rising, fairness checks have quietly gone stale, and four scaled workflows now sit on one vendor nobody priced as a concentration position. The program that looks best on a value-only page is very often the program right before its incident, because the same pressure that produces the flattering number (push volume through, trim the review, ship the next workflow) is the pressure that thins the controls.

Two failure modes, opposite in appearance, invisible on the same page. This is why the strategist's instrument is dual-axis: value and health, reported together, on one page, every quarter. Value tells you what the program produced. Health tells you whether the way it produced it can survive another two quarters of the same.

A program is only as good as the sustainability of the way it is producing its numbers.

The artifact is the AI Value Scorecard: four quadrants (efficiency value, quality value, adoption health, risk posture), each carrying a small number of defined metrics with their source system, their owner, their confidence word, and, above everything, their trend. It is the standing instrument of the program, produced quarterly, and it is the thing your successor will inherit and thank you for.

The Four Quadrants

Quadrants are not categories for tidiness. Each one answers a different question, and each one has metrics that earn their place and metrics that only look like they do. The discipline is as much about what you refuse to print as what you print.

Quadrant one: efficiency value, the money

This is the quadrant executives came for, and the one most likely to be inflated by accident. Three metric families earn a place.

Net cost per unit against baseline. Net, in the Level 3 sense: the verification tax counted. Gate minutes, sampling hours, the rejected-item lane, the exception handling that the new path creates. A gross figure that ignores the cost of checking the machine is not a saving, it is a story. Report cost per unit, not total cost, so volume changes cannot flatter you.

Hours redeployed, with the destination named. When you built the incentive audit you promised the workforce an answer to "where do the saved hours go?" This is where that promise becomes checkable. "3,100 hours saved" is a claim nobody can audit and everybody suspects. "3,100 hours redeployed to the exception backlog and to verification roles" is a claim someone can go and verify by looking at the backlog. If the destination is headcount reduction, say that: a scorecard that hides the honest answer will be believed on nothing else.

Run-rate impact rolled up across scaled items. Here is where portfolios launder estimates into facts, so here is the aggregation rule that keeps you honest. Each scaled item arrives with its own confidence word from its Delta Table: CLEAR (survives every control, mechanism named), PROBABLE (survives the controls but the window limits it), SUGGESTIVE (direction consistent, volume too thin to carry weight). When you add them together, the portfolio total inherits the weakest confidence of its components. One PROBABLE item in the stack makes the total PROBABLE. That rule alone stops the most common quiet fraud in program reporting, which is a total presented with more certainty than any of its parts possess.

Then print two numbers rather than one: the total, and the CLEAR-only subtotal. The CLEAR subtotal is the number a CFO can put in a forecast. The total is the number including the probables. Showing both costs you nothing you should want to keep and buys you the thing that matters most in this room, which is that the next number you present is believed on sight.

What does not earn a place: cumulative savings since inception (it can only go up, so it carries no information), license utilization, seats deployed, and anything ending in "generated."

Quadrant two: quality value, the one most programs omit

Most programs report efficiency and stop. This is a strategic error, because quality value is frequently larger than efficiency value and is trusted more when it is reported well. What earns a place: verified error rate against the pre-AI baseline (and where no such baseline exists, the honest row from Level 3: state the absence as a finding rather than comparing a verified error rate against a rework rate as though they measured the same thing); rework and escape rates; cycle-time tail improvements, specifically the 90th percentile (p90), the boundary of the slowest tenth of items, because customers experience the tail, not the median; and any customer-visible quality signal you already collect, such as complaint volume or dispute rate on the touched process.

Quality value is under-claimed for two reasons. It is harder to monetize (what is one prevented error worth?) and therefore feels weaker in a business case. And it is harder to inflate, which feels like a disadvantage until you notice that finance functions have learned exactly how easy efficiency claims are to manufacture. A CFO who has been shown three cost-saving stories that dissolved under audit will trust a verified error rate that fell from 4.1 percent to 1.4 percent, measured the same way both times, more than a savings figure with a persuasive slide behind it. Difficulty of inflation is a credibility asset. Report the quality quadrant even when you cannot price it, and say plainly which rows are unpriced.

Quadrant three: adoption health, the leading indicator

This is the quadrant that earns the scorecard its predictive power, and the reason value and health belong on the same page rather than in two separate reports read by two separate audiences.

What earns a place: percentage of in-scope work flowing through the new path. Not logins, not sessions, not prompts. Level 1 taught the distinction between activity metrics and value metrics, and this is that rule enforced at program scale: the question is what share of the work that should be going through the redesigned path actually went through it. Then gate adherence and time per review against the standard (if your gate specification says four minutes and reviews are averaging fifty seconds, the gate is a formality and you now know it). Then override rate and its trend, because overrides are the workforce telling you what it thinks of the model's proposals. Then workaround incidence, gathered from floor checks and champion-network signals, counted as open items rather than anecdotes. Then training behavior-present rates from the training blueprint: not who attended, but in what share of observed work the trained behavior is actually present.

Here is the claim that justifies the quadrant's place on an executive page: adoption health leads the value quadrants by roughly one to two quarters. The causal chain is not mysterious. Flow share falls, so fewer items get the benefit. Gate adherence falls, so escapes rise. Workarounds spread, so the process fragments and the redesign's assumptions stop holding. None of that shows up in cost per unit this quarter. All of it shows up next quarter, or the one after. Which means a program whose adoption health is deteriorating while its value numbers hold is not a healthy program with a soft patch. It is a program reporting its own future, and the only people who can act on that information in time are the ones reading this page.

Quadrant four: risk posture, the surprise preventer

The last quadrant exists to make sure nobody in the governance chain is ever surprised, which is a different and more achievable goal than making sure nothing goes wrong.

What earns a place: escapes by severity, with their layer-traced causes (an escape is a defect that reached the customer or the downstream system rather than being caught at the gate); incident count and mean detection latency, the quality-regime key performance indicator (KPI) from Level 3, because how long a problem lived undetected matters more than how many problems occurred; guardrail and tripwire events; fairness-check status by workflow for anything people-adjacent, current or overdue with a date; vendor concentration against tolerance from the vendor register; and compliance-artifact currency against the regulatory calendar, which for European operations means real dates: general-purpose AI (GPAI) obligations live since 2 August 2025, AI-content transparency from 2 December 2026, high-risk Annex III from 2 December 2027.

This quadrant needs something the other three do not: it has to teach its readers how to read it. Executives arrive with an instinct that any non-zero number in a risk column is bad news, and that instinct will destroy your control culture within two quarters if you let it operate. A rising guardrail-event count is usually good news. It means the controls exist, are wired in, and are being exercised by real work. A row of flat zeros is not a comfort; it is a question, and the question is whether the guardrail is functioning at all or simply never tested. So annotate. One line per row, printed on the page:

• Guardrail events, 4 (up from 1). Controls firing as designed on higher volume. Zero would prompt a control test.
• Escapes, 3, all internal-catch. None reached a customer. Causes traced to two layers, both remediated.
• Mean detection latency, 6 days (was 11). Faster detection is the target; the count matters less than the clock.
• Vendor A concentration, 4 scaled items, amber. Above tolerance. A deliberate position with a named review date, not an oversight.

Annotations are not decoration. They are the difference between a scorecard that builds judgment in its readers and one that trains them to punish visibility, and a governance committee trained to punish visibility will get exactly the reporting it deserves, which is the reporting that hides things.

Five Disciplines That Keep the Instrument Honest

Four quadrants is a structure. What makes a scorecard survive contact with an organization is five design disciplines, each of which exists because a specific, predictable thing goes wrong without it.

Metric economy, and the deletion test

Four to six metrics per quadrant, maximum. Twenty to twenty-four in total, and fewer is better. The reason is not aesthetic. A scorecard nobody can hold in their head gets silently replaced in practice by whichever single number an executive happens to remember, and that number will be the wrong one: it will be the biggest, the roundest, or the one that was on the slide when the meeting ran long. A forty-metric dashboard does not give leadership more information. It hands them the job of choosing a summary metric, unsupervised, and they will choose cumulative savings every time.

Apply the deletion test to every metric you inherited: has this number ever changed a decision? Not "is it interesting," not "does someone ask for it," not "is it in the vendor's default report." Has anyone ever done something different because of what it said? If not, it is decoration, and decoration is not free: it consumes the attention budget of the page and dilutes the metrics that do carry weight. Delete it, or move it to the workbook. Most inherited scorecards lose half their rows to this test on the first pass, and read better immediately.

Definitions frozen, sourced, and owned

Level 3 warned about definition drift inside one project. At portfolio scale the temptation is far greater, because a definition change can move a headline number by more than a quarter of real work can, and it can be done in an afternoon by someone with good intentions. So publish, beside the scorecard and not in a separate document nobody opens, three things per metric: the definition in one sentence, the source system, and the named owner. "Gate adherence: share of items requiring gate review where a reviewer decision was recorded before release. Source: workflow event log. Owner: process lead, accounts payable."

Then the rule that does the real work: definition changes are logged and dual-reported for one transition quarter. Old definition and new definition, side by side, once, with the reason for the change stated. It costs one extra column for one quarter and it removes forever the suspicion that a number improved because someone moved the goalposts. Without this rule, you will eventually have to prove a negative in a hostile room, and proving a negative about your own reporting is not a winnable position.

Trends over levels

Print four quarters of trend beside every current value. A verified error rate of 1.6 percent means nothing on its own: it is excellent if last quarter was 3.0 and alarming if last quarter was 1.2. Executives make markedly better decisions from arrows than from absolutes, partly because they have no intuition for what a good absolute looks like in your process and complete intuition for what a worsening line means. The scorecard's job is to make direction unmissable, and the level is context for the direction rather than the other way round.

A practical consequence: this constrains your metric set even further, because a metric you cannot produce consistently for four quarters is not scorecard material yet. Put it in the workbook, build its history, and promote it when it has a trend to show.

Confidence words carried through

Every value metric carries its CLEAR, PROBABLE, or SUGGESTIVE word from its underlying Delta Table, printed on the page. This is the single mechanism that prevents the scorecard from doing what most program reporting does, which is to launder an estimate into a fact by moving it up a level of aggregation. An estimate travels upward through three decks and arrives in the board pack with the same typography as an audited figure. The confidence word travels with the number and refuses that promotion.

The strategist's costume test is simple and slightly painful: are you willing to mark your own headline PROBABLE when the rules say so? If yes, the words mean what they say and the whole page inherits that credibility. If you have never printed anything but CLEAR, nobody in the room believes any of them.

The one-page constraint, and the workbook behind it

The scorecard is one page. Not one page plus an appendix that is really the report. One page, four quadrants, current value, four-quarter trend, confidence word, one-line annotation where a row needs teaching. Behind it sits the workbook: definitions, sources, queries, the Delta Tables of every scaled item, the cost models, the fairness-check evidence, the vendor register extract. The workbook can be two hundred pages and nobody minds, because its job is to be complete and available, not to be read.

This is the compression discipline from Levels 2 and 3 at its final scale, and the arithmetic is the same as it was for the heat map and the business case: the audience's attention is fixed, so length is not a measure of thoroughness but a way of spending a fixed budget badly. If it will not fit, the problem is metric economy, not page size.

Who produces it, who validates it, who consumes it

Three roles, deliberately separated. The strategist produces the scorecard and owns its structure. An independent party validates the value numbers, per the measurement-independence rule: whoever is accountable for the program's success does not get to be the sole source of the evidence that it succeeded. In practice this is a finance analyst or an internal audit partner who can re-run the cost model and the flow-share query against source systems. The governance committee consumes it, on a standing agenda item, with the authority to act on the health quadrants and not merely admire the value ones. The next lesson takes this page into the C-suite as a narrative; the lesson after that gives the committee its operating design.

Worked Example: One Quarter on One Page

Here is the running enterprise storyline's Q3 scorecard. Every figure is hypothetical and exists to show the shape of the instrument, not to serve as a benchmark. The program has four scaled items and two in pilot; the invoice-exception workflow is the largest component.

Now read it the way the instrument is meant to be read, quadrant against quadrant rather than row by row.

The efficiency rows carry the two-number honesty. The portfolio shows $412,000 of annualized run-rate impact, and the page immediately splits it: $260,000 CLEAR, $152,000 PROBABLE. The total inherits PROBABLE because its weakest component is PROBABLE. A CFO reading this can forecast against 260 and treat 412 as an upside case, which is precisely the decision she wants to make and precisely the one a single 412 would have prevented her from making knowingly. Meanwhile net cost per exception has fallen from $31.20 to $22.90 over four quarters, and it is per-unit and net, so a volume swing cannot flatter it and the cost of checking the machine is already inside it.

The quality rows include the honest absence. Verified error rate is 1.4 percent and the note refuses the flattering comparison: the legacy process never verified its own outputs, so its 12 percent rework rate is a different measurement, not a baseline. Printing that caveat on the executive page costs a dramatic-sounding delta and buys the right to be believed about the p90 improvement from 9.1 to 4.0 days, which is CLEAR, has a named mechanism, and is the row customers can actually feel.

The adoption rows are the reason to be calm. Flow share at 84 percent and rising, gate adherence at 91 percent with time per review holding at 3.8 minutes against a four-minute standard, overrides falling, two open workarounds. Health is improving alongside value, which means the value is not being purchased by thinning the controls. Had gate adherence been falling while cost per unit improved, the correct reading of this page would be the opposite of the comfortable one.

The risk rows contain the only amber and the only overdue item, both named. And here is the whole page compressed into the sentence you say out loud before anyone else characterizes it: "Value is real and improving; the single amber is a concentration position we are choosing deliberately and reviewing in January; the one overdue fairness check has an owner and a date." One line, three clauses, no surprises available to anyone in the room. Write that line yourself, every quarter, before the meeting. If you cannot write it honestly, you have found this quarter's real agenda item.

The Program That Reported One Number

A media company runs a respectable AI program: content tagging, rights clearance support, and an advertising-operations workflow. Every quarter, the program reports to the board with one figure, "cumulative annualized savings," and for five straight quarters that figure rises pleasingly: 300k, 640k, 1.1m, 1.6m, 2.1m. The board is pleased. The program lead is promoted once. Nothing on the reported page covers adoption, escapes, or control health, because nobody asked for it and the one number was going the right way.

In quarter six a quality incident surfaces in rights clearance. It had been building for eleven weeks. The reconstruction afterward is brutal in its simplicity: under a throughput push tied to a launch, gate adherence in that workflow had fallen from 88 percent to about 40 percent, escapes had tripled, and the savings number had kept rising precisely because reviews were being skipped. The saving and the incident were not two events. They were one event, reported once, with its favorable half on the slide and its unfavorable half in a system nobody was reading.

The board's response is not proportionate to the incident, which is contained within a month and costs less than a quarter of the reported savings. It is proportionate to the surprise. Five quarters of a rising line had taught the board that this program was understood and safe, and the incident taught them that it was neither, which retroactively devalues every number that came before it. Funding is halved. The program lead's credibility does not recover in that organization.

The lesson people usually draw is that the scorecard was missing a metric, and that gate adherence should have been added. That reading is too small. The page did not fail to include a row. It was structurally incapable of showing the tradeoff the program was actually making, because it had one axis. A single-axis instrument cannot represent "we bought this number with that control," and buying numbers with controls is the most common thing a program under pressure does. Add gate adherence to a one-number report and it becomes a two-number report that still cannot show the exchange rate between them. What shows the exchange rate is value and health printed side by side, quarter over quarter, so that a rise in one beside a fall in the other is visible as a single fact rather than as two facts filed in different places.

What to Do Monday Morning

The scorecard is not a document you commission. It is one you draft badly this week and improve every quarter for the rest of the program.

1. Draft the four quadrants with no more than six metrics each. One page, on paper if that helps. Efficiency value, quality value, adoption health, risk posture. If a quadrant is empty, that emptiness is the most useful thing you will learn on Monday: most programs cannot fill quality value or adoption health, which is exactly why they are surprised later.
2. Run the deletion test on every metric you inherited. For each one, name the decision it has changed in the last year. If you cannot name one, delete it or demote it to the workbook. Expect to remove half. The page gets shorter and its signal gets stronger in the same stroke.
3. Publish definitions, sources, and owners beside the numbers. One sentence, one system, one name per metric. This takes an afternoon and it is the single highest-return afternoon in the quarter, because it is what makes next year's comparison meaningful and next year's argument about the numbers short.
4. Add four-quarter trends and confidence words. Reconstruct backwards where you can, and mark the gaps honestly rather than filling them. Apply the aggregation rule to any roll-up: weakest confidence wins, and print the CLEAR-only subtotal beside the total.
5. Write the one-line reading of your own page before anyone else does. One sentence naming what is real, what is amber, and what is overdue with an owner and a date. Then find the person who will validate the value numbers independently, and book them for next quarter before you need them.

With the instrument built, the remaining problem is not measurement but translation: how this page becomes a quarterly narrative that a chief executive and a board can act on without being either alarmed or lulled. That is the next lesson.

Key Takeaways

• Reject the single program ROI number: it is blind to both diseases that kill programs, stalled value (McKinsey's 88 percent using AI against roughly 39 percent seeing any EBIT impact) and hidden risk (real value produced on an eroding control base).
• Build the AI Value Scorecard on two axes and four quadrants: efficiency value, quality value, adoption health, and risk posture, printed together on one page every quarter.
• Report efficiency net, per unit, and honestly aggregated: the portfolio total inherits the weakest confidence of its components, and the page shows both the total and the CLEAR-only subtotal a CFO can bank.
• Claim quality value even when you cannot price it, because verified error rates, escapes, and p90 tail improvements are harder to inflate, and that difficulty is exactly why finance trusts them more.
• Treat adoption health as the leading indicator it is: flow share, gate adherence and review time, override trend, workaround incidence, and behavior-present rates predict the value quadrants by one to two quarters.
• Annotate the risk quadrant so it teaches its readers: rising guardrail events usually mean the controls are working, while a row of flat zeros is a question rather than a comfort.
• Enforce the five disciplines (metric economy with the deletion test, definitions frozen and sourced with changes dual-reported for a quarter, trends beside every level, confidence words carried through, one page with a workbook behind it) and separate the roles: strategist produces, an independent party validates the value numbers, governance committee consumes with authority to act on health.
• Remember the single-number program: it did not omit a metric, it was structurally unable to show that its savings and its incident had the same cause, and the board punished the surprise rather than the incident.