The AI-Ready Data Program: From 63% Problem to Asset
The quarterly data report ran to one line, and the line was a joke nobody laughed at: the vendor master contains 312 duplicate records. Same number as last year. Not approximately the same. The same. Ten months earlier a contractor team had removed 312 duplicates from that table, been thanked in a steering-committee slide, and gone home. The records in the file now were different duplicates, created by different people through the same unchanged onboarding form, at almost exactly the historical rate, because nothing about how vendor records get created, owned, or checked had been touched. The cleanup was real. The clean state lasted about six weeks. This lesson is about the difference between a project that cleans data and a program that keeps it clean, and why only the second ever shows up in your AI results.
The Cleanup That Came Undone
Start with the statistic that governs this chapter, and read it more carefully than most people do. Gartner reported in February 2025 that 63 percent of organizations either lack AI-ready data practices or are unsure whether they have them, and forecast that through 2026, 60 percent of AI projects without AI-ready data will be abandoned. Nearly everyone quotes that as a warning about data quality. Look at the noun: practices. Not data. Not quality. Practices. Gartner is describing missing machinery, not a pile of dirty records, and a dirty pile calls for a cleanup while missing machinery calls for a program. The two look identical on a funding request and completely different eighteen months later.
The failure story: six clean months, then the slow return
Here is the failure in full, with illustrative numbers. A distribution business funds a six-month data quality initiative as a capital project. The work is competent: five contractors profile nine source systems, deduplicate customer and vendor masters, standardize address and unit-of-measure formats, backfill product attributes, and hand over a defensible clean state on time. Then the team disbands.
Nothing else changed. No business leader was accountable for those tables staying fit. Nobody had budgeted hours to profile them again. The onboarding forms, the free-text fields, the three ways a branch can enter a unit of measure: all untouched, still producing the defects the contractors had removed. And no written standard for "clean enough" existed, so nobody could detect the state degrading.
Eighteen months later the duplicate rate has returned to roughly 71 percent of its pre-project level. Two AI pilots built on the clean state have quietly degraded: the demand forecast that was accurate in month three is now politely ignored by the planners, and the supplier-spend tool produces a number the category managers no longer trust. This is the Level 3 drift problem one layer lower than most teams watch: not model drift but data drift underneath the model, same symptom, a tool that was right once and is wrong now while nobody can say when it turned.
The most expensive consequence is not technical. It is the sentence now lodged in the CFO's head: data quality work does not stick. She has direct evidence: she funded it, it worked, and it came undone. The second cleanup will be harder to fund than the first, the third impossible. The organization has not just failed to fix its data; it has spent the credibility it needed to try again.
Projects clean data. Programs keep it clean. The difference between them is owners with budgeted hours.
A condition, not an event
Data unreadiness is a condition, not an event. An event happened once and can be undone. A condition is a state the organization reproduces daily through normal operation: every vendor entered through an unvalidated form, every branch spelling the same customer three ways, every field left free-text because constraining it would have annoyed someone in 2019. Not residue, production. Cleanup is a one-time subtraction against a continuous addition. Emptying the bucket is necessary and does nothing about the rain. A program is the roof.
You have already done the two things that come before the program. The enterprise audit built a Critical Dataset Register: 31 datasets that survived criticality triage out of roughly two thousand, scored on five dimensions, producing a 22-line remediation backlog of about 210 effort-days sorted by beneficiaries per effort, with the honest headline that only 7 of the 31 were AI-usable as they stood. The roadmap then took the top lines into wave 0 and funded them. Both were correct; neither is durable. The audit is a photograph of a moving thing and wave 0 is a single push. Missing is the machine that keeps taking photographs and keeps pushing.
The artifact: the AI-Ready Data Program Charter
The named artifact you leave holding is the AI-Ready Data Program Charter: four pages that convert a remediation backlog into a standing capability. Four components, and four because each fails on its own:
- The working definition. Testable criteria for what "AI-ready" means, per dataset class. Without it, readiness is an opinion and every argument about it is unwinnable.
- Ownership that survives. An accountable owner and a funded steward per registered dataset, the steward's hours priced as a budget line. Without it, the definition is a document nobody must satisfy.
- A prioritized work queue. Fed by portfolio demand and certification lapses, published where functions can see it. Without it, stewards work on whatever is most interesting or most loudly requested.
- A progress metric. One headline number leadership watches quarter over quarter. Without it, the program cannot prove it works and loses its funding to something that can.
BCG's 10-20-70 rule (10 percent of the effort in algorithms, 20 percent in technology and data, 70 percent in people and process) is usually quoted at model builders, and applies with more force here. A data program is almost entirely the 70 percent: definitions, accountabilities, queues, and a number on a page. There is very little technology in this lesson, and that is the point.
Component One: The Working Definition Nobody Writes
This component gets skipped, almost every organization, almost every time. Teams write the charter, name owners, stand up the queue, launch the dashboard, and never write down what "AI-ready" means in testable terms. The consequence arrives four months in, when an analytics lead says the customer master is ready, a data engineer says it obviously is not, and no instrument in the room can settle it. That argument is not about facts but about a missing definition, and it repeats monthly forever.
So write it. You are not inventing criteria: the five dimensions you scored in the audit convert into five pass/fail tests. Scores diagnose; tests govern. A dataset does not have a readiness score of 3.4 in the program's language. It passes or it does not. Watch test 5 in particular: "can we use this data for that?" is usually a three-week legal question whose answer is never written down, so the next team asks again, and a standing determination converts that recurring tax into a one-time cost.
| Test | Passes when | Evidence that satisfies it |
|---|---|---|
| 1. Owned and budgeted | A named business owner is accountable for fitness and a named steward has budgeted hours against it | Owner name in the register, steward hours in an approved cost centre |
| 2. Profiled and published | Profiled within the last N months (12 default, 6 for tier-1), quality figures published where consumers read them | Dated profile report: completeness, duplicate rate, codebook conformity |
| 3. Accessible within SLA | An approved consumer obtains working access within the published service-level agreement (SLA) | Access-ticket log: median and 90th-percentile grant time inside the SLA |
| 4. Monitored with a named recipient | Lineage documented and freshness monitored, alerts routed to a person, not a distribution list | Monitor configuration, recipient's name, an example of an alert being actioned |
| 5. Usable under policy | An approved consumer class can use it under existing policy without a per-request legal review | A standing determination covering purpose, personal data handling, contract terms |
Ready for whom: the consumer-class clause
A dataset is never AI-ready in the abstract. It is ready for a named consumer class, and the phrase belongs inside the definition. The service-history table is comfortably ready for a retrieval assistant answering "what did we do for this customer last time," where a missing field costs a human ten seconds. The same table is not remotely ready for an automated warranty-eligibility decision, where a missing field produces a wrong denial, a complaint, and possibly a regulatory question. Same table, same day, same figures, two verdicts. Fitness is a relationship between a dataset and a use, as fitness for purpose has always worked in quality management: a tolerance fine for a bracket is not fine for a bearing.
So the register carries a consumer class against each certification. Three cover most enterprises: human-in-the-loop assistive (a person reviews the output before it has any effect), analytical (reporting and forecasting, where record errors wash out but systematic bias does not), and automated decisioning (the output acts unread). The tests stay the same; the thresholds tighten down the list. A 2 percent duplicate rate is invisible to the first class, annoying to the second, unacceptable in the third.
Certification: how readiness becomes a register column
The definition acquires teeth through a small bureaucratic act: certification. A dataset is certified against the standard for a consumer class, on a date, by a named certifier, and the certification expires: annually by default, six-monthly for tier-1 datasets feeding automated decisioning.
Certification does three things nothing else does. It converts readiness from a feeling into a column anyone can read. It creates an expiry, so neglect becomes visible on a schedule instead of through a failing pilot. And it gives a project team a fast planning answer: this dataset is certified for analytical use until March, so build your forecast on it; that one is uncertified, so your plan needs a remediation line and a date. One rule stops it rotting into a rubber stamp: the certifier is not the owner. The owner attests, the program certifies, the same separation you built into stage-gate reviews. People are generous about work they are accountable for, through familiarity rather than dishonesty.
Component Two: Ownership That Survives the Org Chart
The owner attests; the steward works
Every organization that has run a data initiative has a spreadsheet with an "owner" column, and almost none have ownership. That gap comes from collapsing two different jobs into one word.
The accountable owner is a business leader who owns the dataset's fitness for its consumers, usually in the function that creates the data, because creation is where fitness is determined: Procurement owns the vendor master, Field Service owns the service-history table. The owner does not profile records. The owner decides what "fit" means, signs the annual attestation, settles disputes about definitions, and absorbs the consequence when their data blocks someone else's funded work. Ownership without consequence is a name in a spreadsheet.
The operating steward is the person with budgeted hours who does the work: runs the profile, maintains the codebook, fixes the defect batch, answers the "what does this field mean" question, watches the freshness monitor. The steward usually sits outside the owner's reporting line, and is the program's actual capacity.
Most organizations name owners and skip stewards, which is backwards in terms of what fails first. An owner without a steward is a leader accountable for work nobody has time to do, and the rational response, visible within a quarter, is to stop attending. The steward's hours are the program's real budget line; everything else in the charter is paper.
The arithmetic: pricing the stewardship
You cannot defend a budget line you have not calculated. Tier the 31 registered datasets by portfolio dependency, not size. Tier-1 are those two or more funded portfolio items depend on, or that feed any automated-decisioning consumer: here, 8 of 31. Tier-2 is the remaining 23: registered, certified, watched, not currently load-bearing.
| Tier | Datasets | Steward load each | Subtotal | What the hours buy |
|---|---|---|---|---|
| Tier 1 | 8 | 0.2 FTE (about one day a week) | 1.6 FTE | Six-monthly profiling, active defect work, project support, monitor response |
| Tier 2 | 23 | Shared pool | 1.0 FTE | Annual profiling, certification upkeep, triage of incoming questions |
| Total standing stewardship | 2.6 FTE | Roughly 310,000 per year fully loaded (illustrative) | ||
FTE means full-time equivalent: 0.2 FTE is one day a week of a real person's time, protected in their objectives, not squeezed around their day job. Stewardship living in the gaps of a calendar disappears the first busy month.
Defend the number as a portfolio line, not a hygiene request, exactly as you defended a capability build in the roadmap lesson. The sentence that works is arithmetic, not virtue. Seven of the eleven funded portfolio items depend on tier-1 datasets. This 310,000 line protects the data those seven items run on, and its absence is what the 60 percent abandonment forecast describes. Compare "we would like to invest in data quality," which has never survived a finance review. One is insurance on committed spend with a dependency count; the other is a preference.
When an owner declines the role
They will, roughly one in eight, and mostly from the functions whose data matters most, because those are the busiest functions. Three flavours, one real objection. Capacity refusal ("I cannot take this on this year") is a steward problem in an owner costume: show them the steward line and an attestation calendar costing two hours a year. Legitimacy refusal ("this is not our data, we just enter it") is often correct, and tells you the register assigned ownership to the wrong function. Political refusal ("we are not signing anything that makes us liable for what other teams do with our records") is the real objection, and the strategist cannot resolve it: you cannot appoint owners, and pretending otherwise burns the relationship.
What you have is an escalation path, named in the charter before you need it. Unresolved assignments go to the governance forum with a one-page brief: dataset, proposed owner, refusal, portfolio items affected, recommended resolution. The forum decides, because a cross-functional accountability dispute is exactly what a governance body exists to settle; later in this level you build that body, and the charter reserves the seat. Write the briefs as decision requests with options, never complaints, and always include the honest option of moving ownership to the function that consumes the data most.
Component Three: The Queue That Shrinks Itself
The intake rule
The third component decides what the funded stewards work on, and it is where good intentions quietly betray the portfolio. The audit's backlog was sorted by beneficiaries per effort: how many pipeline use cases a fix unblocks, divided by the effort-days it takes. Promote that one-time sort to a standing intake rule, in the charter, in one sentence: work enters the queue when a portfolio item depends on it or when a certification lapse is detected, is ordered by beneficiaries per effort, and enters no other way without a governance decision.
Both triggers matter. The first ties the program to committed demand, which keeps it fundable: every active line traces to a funded initiative. The second is maintenance, and it is what makes this a program rather than a demand-response desk. A tier-1 dataset whose certification expires in six weeks generates a queue line automatically, before anything breaks and before anyone complains. That is the roof, working in the dry season.
The anti-pattern this prevents is not a story about bad people. Give a competent data team an unsorted list and they gravitate to the technically interesting problem (the lineage graph nobody has built) and the loudest requester's problem (the executive who mentioned it in a corridor). Both feel like progress; neither correlates with portfolio value, and the boring fix that unblocks four funded initiatives sits at line 14 because nobody is excited by unit-of-measure conformity.
Then publish the queue: same page as the roadmap, monthly, each line showing dataset, beneficiary count, effort, steward, and position. A function can then see where its data sits and stop asking, and owners generating the most lines feel public pressure.
The leverage move: fix the creation side
Cleaning is downstream work: it subtracts defects the organization is still producing. The durable win is upstream, at the moment the record is created. The free-text field that gets a codebook and a dropdown. The onboarding form that fuzzy-matches the vendor name before it lets you save. The job-completion screen that will not submit without the parts-used field.
So attach a standing question to every recurring line: what upstream change would retire this line forever? Ask it at every queue review and record the answer even when the answer is "nothing cheap." Some lines cannot be retired, because external data arrives how it arrives. Most can, and the cost asymmetry is dramatic. The vendor-master dedupe line runs at roughly 6 effort-days per quarter, forever, about 24 days a year; the creation-side fix, a validated onboarding form with duplicate detection at entry, was estimated at 12 IT-days once. It pays back inside two quarters and corrects the data while a human is present, the only moment correction is cheap.
State the ambition in the charter, because it inverts how programs behave: the program's job is to make itself smaller. A queue that grows every quarter has accepted the condition; a queue retiring recurring lines one by one is changing the rate at which the organization manufactures its own problem, which is the only progress that compounds.
Discipline one: one plan, not two organizations
The data program's queue and the roadmap's waves must be one plan reviewed together. Two plans produce the negotiation that has wasted more transformation quarters than any technical problem: the initiative team says the data was supposed to be ready, the data team says nobody told them the date moved, both are telling the truth, and the wave slips. The mechanism that prevents it is unglamorous: the steward lead attends gate day, register open, so a readiness answer is a certification status and a queue position with a date rather than a project manager's assurance, and a moved wave date re-sorts the queue in that meeting rather than by email three weeks later. The reciprocal obligation matters as much: no portfolio item passes its readiness gate depending on a dataset that is neither certified nor holding a funded queue line dated before its build start. Enforced at the gate, that rule turns the data program from a service desk into a control.
Discipline two: resisting enterprise-catalog gravity
The audit lesson warned about boiling the ocean: cataloguing all two thousand datasets before doing anything. Most teams accept that at audit time and meet the temptation again, more seductively, about three quarters into a successful program. Success creates the gravity: the program works, functions notice, and requests arrive to catalogue theirs too, certify everything, build the glossary. Friendly requests destroy focus more effectively than any opposition, because coverage of 31 well-chosen datasets is a program and coverage of 400 is a documentation project with no consumers.
The answer is not "no." It is the register's demand-derived admission criteria, revisited quarterly as the portfolio grows: a dataset enters when a pipeline item depends on it, so the honest answer is bring a use case into the pipeline and your dataset comes with it. A register that grows on request grows forever, and the 2.6 FTE adequate for 31 datasets is visibly inadequate for 90.
Component Four: The Number Leadership Watches
The fourth component exists because of a hard fact: a program that cannot show movement loses its funding to one that can, regardless of which does more good. You need one number that moves quarter over quarter and that a CFO reads in four seconds: AI-Ready Coverage, certified datasets as a percentage of the register, weighted by portfolio dependency.
The weighting is not a nicety; it prevents gaming. Unweighted, the fastest way to raise coverage is to certify the eight easiest datasets, easy precisely because nobody depends on them. Weight each by the number of funded portfolio items depending on it and the incentive inverts: certifying the vendor master, which four initiatives wait on, moves the number far more than three quiet reference tables. An illustrative trajectory: 23 percent at charter approval (the audit found 7 of 31 datasets AI-usable), rising to 61 percent over three quarters as wave-0 remediation completes. Coverage moves in steps and then flattens, because the easy tier certifies first.
Two integrity checks, not a dashboard
A single headline number is manipulable, so carry exactly two supporting lines. Not twelve. Two.
- Access-SLA attainment: the percentage of access requests granted inside the published SLA, with median grant time. This catches a program certifying datasets that remain unreachable in practice, and it is the line business users feel directly. Here the median was 14 days at audit, so every pilot's first three weeks went on waiting for permission.
- Mean age of certifications: the average time since each was issued. This catches the opposite failure, a program coasting on old certificates: rising coverage with rising certificate age means the number is drifting away from reality, which is exactly the condition the 63 percent describes.
Three lines total, deliberately. Forty-tile data-quality dashboards are read by nobody outside the team that built them, because a leadership audience cannot tell which number should change its behaviour. One number and two integrity checks gets asked about every quarter, which is what keeps a program alive between crises. Put it beside the readiness heat map and the value scorecard: the fastest way to lose the competition for attention is to hold your own meeting.
A worked example: the first quarter of a program
Take the 2,400-person B2B services and distribution business from the audit and the roadmap: six functions, 31 registered datasets, a vendor master carrying 312 duplicates and no owner since a 2022 migration, a 14-day median access-grant time. One quarter of standing up the program, all figures illustrative.
Month one, the charter. Four pages approved by the steering committee: the five-test definition with three consumer classes, the ownership model, the intake rule, the coverage metric. Two workshops produced one argument worth having, about whether "profiled within 12 months" was too lax for the vendor master. It was; tier-1 became six months.
Months one to two, ownership. All 31 datasets assigned an owner and a steward. Four owners refused; three resolved directly (two capacity refusals dissolved by the steward line and the two-hour attestation, one legitimacy refusal where the register had guessed wrong). The fourth went to governance and was resolved honestly rather than politically: ownership of the pricing-exception table moved from the finance team that inherited it in the migration to the commercial function that consumes and cares about it. That was the quarter's most useful hour, because it ended a three-year-old ownership fiction.
Month two, funding. Stewardship approved at 2.6 FTE, about 310,000 per year fully loaded, defended with the dependency arithmetic: 7 of 11 funded portfolio items depend on tier-1 datasets. Finance booked it as a portfolio line rather than overhead, which matters more than it sounds. Overhead gets trimmed in a bad quarter; portfolio lines get trimmed only by killing the initiatives they support.
Months two to three, the queue. Wave-0's vendor-master remediation completed: duplicates resolved, an owner in place, the table certified for analytical and assistive use. Then the move that proved the program was a program. The queue review asked the standing question, and the creation-side follow-on entered as its own line: a validated onboarding form with duplicate detection at entry, 12 IT-days, retiring a recurring dedupe line of roughly 6 days per quarter. One sentence in the queue log was worth more than the dedupe itself: this line is now closed permanently rather than scheduled again.
End of quarter, the number. AI-Ready Coverage moved from 23 percent to 39 percent. Access-SLA attainment went from unmeasured to 78 percent, median grant time falling from 14 days to 2.8, almost entirely by pre-approving standing consumer classes instead of reviewing each request. Mean certification age: 41 days, meaningless now and decisive by quarter five.
Read the quarter honestly: coverage did not reach 61 percent and was never going to. What happened is that a snapshot became a machine, with 31 datasets carrying owners who will be asked to attest, stewards with protected hours, a queue that reorders itself against portfolio demand, and one recurring line retired for good. The distributor bought a clean state; this organization bought the capability to keep producing clean states, which is the only purchase that survives eighteen months.
One honest framing for the sceptics on your steering committee: this does not guarantee AI success. MIT's 95 percent of enterprise GenAI pilots producing no measurable profit-and-loss return, and McKinsey's 88 percent of organizations using AI while only about 39 percent report any EBIT impact, are workflow and measurement stories more than data stories. What the program removes is one well-documented cause of abandonment, the one Gartner sized at 60 percent of projects lacking AI-ready data. It buys you the right to fail for more interesting reasons. It also needs governance that clears paths instead of gating them, which is the next lesson.
What to Do Monday Morning
The charter is four pages and the first draft takes a focused day. Here is the order that works.
- Write the five-test definition for your top dataset class. One page, pass/fail statements, no scores, starting with the class your most-funded initiative depends on. Set the profiling interval and the access SLA explicitly: those are the two numbers people argue about later, and you want them written before there is a dispute to lose.
- Name the consumer classes: human-in-the-loop assistive, analytical, automated decisioning. One sentence per class on what an error costs, because that sentence justifies the tighter thresholds.
- Assign owner and steward separately for your tier-1 datasets, and price the steward hours. Two columns, two names, an FTE figure per dataset. Take the total to your sponsor as a portfolio line with the dependency count attached, and do not soften it; a stewardship line quietly halved fails slowly and expensively.
- Publish the remediation queue with its beneficiaries-per-effort sort, wherever the roadmap already lives, including the lines you are not working on and their position. The transparency costs an uncomfortable week and buys a year of not being asked.
- Pick one recurring cleanup line and find its creation-side fix. The one scheduled more than twice. Estimate the upstream change, compare it against the annual recurring cost, and put it in the queue as its own line with a named IT owner. Retiring one line permanently is the most persuasive thing this program does in its first quarter.
- Put AI-Ready Coverage on the quarterly page, weighted by portfolio dependency, with access-SLA attainment and mean certification age underneath it, beside the heat map and the value scorecard.
- Book the steward lead into gate day, a standing invitation starting with the next one. It is the cheapest item here and the one that keeps the queue and the roadmap from becoming two plans.
Key Takeaways
- Read Gartner's 63 percent as a statement about missing practices, not dirty data: the same research puts 60 percent of AI projects without AI-ready data on the abandonment path through 2026, and unreadiness is a condition the organization reproduces daily, not an event to be undone once.
- Convert the audit's backlog into a standing capability with the AI-Ready Data Program Charter: a working definition, ownership that survives, a prioritized queue, and a progress metric, four components because each fails alone.
- Write the definition as five pass/fail tests per dataset class, then certify against it: owned and budgeted, profiled and published, accessible within SLA, monitored with a named alert recipient, usable under policy without per-request legal review. Scores diagnose, tests govern, and a dated certification (for a named consumer class, by a certifier who is not the owner) turns readiness into a register column rather than an opinion.
- Split the accountable owner from the operating steward and fund the steward's hours: 8 tier-1 datasets at 0.2 FTE plus 23 tier-2 sharing 1.0 FTE gives roughly 2.6 FTE, defended as a portfolio line with the dependency count (7 of 11 funded items).
- Feed the queue only from portfolio demand and certification lapses, sort by beneficiaries per effort, and publish it: otherwise stewards drift to the most interesting problem or the loudest requester while the initiative needing unit-of-measure conformity waits at line 14.
- Ask what upstream change retires each recurring line forever: 12 IT-days for a validated onboarding form beats 6 effort-days of dedupe every quarter forever, and a program whose job is to make itself smaller is the only kind that compounds.
- Report one weighted number with two integrity checks: AI-Ready Coverage beside the heat map and value scorecard, with access-SLA attainment and mean certification age beneath it, because a forty-tile dashboard is read by nobody who can fund you.
Skill.re