←
AI-Ready Foundation
Proficient · M4 · lesson 4 of 5 · queued
Preview — browse every lesson free. Enroll to mark lessons complete, open partner links and save your progress. Login & enroll →
Data Architecture for AI: Data Contracts, Semantic Layers, and Knowledge Graphs
📖
now learning

Data Architecture for AI: Data Contracts, Semantic Layers, and Knowledge Graphs

15 min

"How many active customers did we have last quarter?" The sales director types it into the company assistant and gets an answer at once: a confident number that does not match the board pack. Nobody can say whether it read the wrong table, used the wrong meaning of "active", tripped on a renamed column, or picked up an old definition from a slide deck nobody owns. The data team loses two days tracing it, and the director quietly stops using the tool. This lesson is about data architecture for AI: a small set of written agreements (data contracts, a semantic layer, lineage, named owners and review dates for the documents the assistant reads and, later, a knowledge graph) that tell a model what your data means, so that the next answer to that question matches the board pack.

You are at Level 3 of the ladder, AI Operational, in the second of five lessons. Lesson 11 on shipping, monitoring and rolling back AI in production gave you a release gate, a watch on live quality and a rollback path, so you can ship an AI system and take it back. This lesson fixes what that system reads. Lesson 13 on defending against prompt injection and oversharing then asks the security question about the same data: who and what else can the assistant read, and can the text it reads give it orders.

The Problem: Your Data Has No Labels, and the Model Is Guessing

What did GoCardless do about data that kept breaking?

GoCardless introduced data contracts in 2022, one stream at a time, and made them self-serve for the engineers who produce the data. The account comes from Monte Carlo's write-up "7 Lessons From GoCardless' Implementation of Data Contracts" (first published 7 July 2022, updated 30 March 2026), which relays Andrew Jones's lessons from the rollout.

The situation will sound familiar. One team produces data, other teams consume it, the producer changes something sensible, and something far away breaks, because nobody had written down who relied on what.

The GoCardless answer was deliberately small. A data contract there defines the schema, categorises the data, assigns ownership and specifies the services that need it. Engineers write the contract as code (in a configuration language called Jsonnet), merge it through GitHub like any other change, and the merge automatically deploys the BigQuery and Pub/Sub resources that carry the data. Writing the contract is how you get your pipeline.

About six months in, the write-up reports, roughly 30 contracts were deployed, powering about 60% of the company's asynchronous inter-service events. Jones's lessons are worth reading slowly, because each one cuts against a common instinct:

  • Contracts rarely change once set. Endless renegotiation, the usual fear, did not happen.
  • Make self-service easy. If the contract is the fastest route to a working pipeline, people write contracts.
  • Introduce contracts at moments of change. New streams got contracts. Stable services were not dragged through a migration.
  • Build categorisation and governance in early. Classifying inside the contract beats bolting it on later.
  • Service-to-service teams adopt first. Start where the pain of a broken dependency is felt most directly.

Three honest caveats before you quote this in a steering committee. The figures are from 2022, with no newer numbers found. The source is a vendor blog relaying a customer's account. And the case is about data engineering reliability, not a measured gain in AI accuracy. What it teaches is the shape of the fix: a data contract is not a documentation project but a short written agreement that lives next to the code, is checked by a machine, and arrives one stream at a time when something is changing anyway.

What does missing meaning cost when a model reads your database?

It costs you correct answers, and the published numbers are blunt. I will not give you a currency figure, because the widely quoted ones are undated or unsourced. What can be sourced is the rate of wrong answers on text to SQL, the task of turning a plain-language question into a database query.

In a benchmark by Sequeda, Allemang and Jacob of data.world (arXiv:2311.07509, 13 November 2023), GPT-4 answering questions zero-shot, directly over an enterprise SQL schema, reached 16% accuracy. Over a knowledge-graph representation of the same database it reached 54%. Same model, same database, more than three times the accuracy once meaning was supplied. (The authors work for a vendor in this field; the paper is public and checkable.)

A newer study, Rumiantsau and Fokeev (arXiv:2604.25149, 28 April 2026), put 100 natural-language questions over a retail dataset to three current frontier models. With the schema only, they scored 45.5 to 50.5%. With one hand-written semantic-layer document of about 4 KB added, they scored 67.7 to 68.7%, a gain of 17 to 23 points that the authors report as statistically significant (p < 0.01). Within each condition the three models were statistically indistinguishable. The authors appear to be vendor-affiliated, so read this as indicative, not independent proof.

Now read the half people leave off the slide. In the best condition accuracy was 54% in one study and about 68% in the other, so between 31% and 46% of answers were still wrong. Supplying meaning helps a great deal. It does not remove the need for a human to check.

What goes wrong when the assistant reads documents instead of tables?

The source goes stale and nobody notices, because nobody owns it. At Level 3 most assistants spend more of their day in policy libraries, price lists and wikis than in databases, and there the usual failure is a missing owner. Nobody is accountable for whether the returns policy in the library is the current one, so three versions sit side by side and the assistant reads whichever the search ranks first.

Two sources frame the risk. Barnett and colleagues at Deakin University (arXiv:2401.05856, January 2024), cataloguing how retrieval systems fail, name "missing content" as the first failure point: a question that cannot be answered from the available documents, where the system can still be fooled into giving a response. Their fourth, "not extracted", is an answer that sits in front of the model but is not pulled out, which they say typically happens when there is too much noise or contradicting information in the context. Two live versions of one policy are contradicting information by definition.

The second comes from a vendor whose assistant reads such libraries. Microsoft's deployment guidance for Microsoft 365 Copilot (Microsoft Learn, updated 28 October 2026) tells you to find sites and files that are "overshared, ownerless, inactive" before rollout, to "assign or confirm site ownership", and to keep addressing "inactive or obsolete sites" afterwards. Oversharing is Lesson 13's subject; ownership is this one's.

The liability does not move to the software: in Moffatt v. Air Canada (BC Civil Resolution Tribunal, 14 February 2024) the tribunal held the airline responsible for all the information on its website, chatbot included, a case Lesson 8 on building a golden test set tells in full. Whatever the assistant reads, you are publishing.

Why do capable organizations keep getting this wrong?

Four habits, and you will recognise at least two.

  1. They treat a wrong answer as a model problem. The reflex is to shop for a better model. Both benchmarks point the other way: the added context made the difference.
  2. Business meaning lives in heads and dashboard formulas. The schema says cust_status = 'A'. Only the finance analyst knows that "active" also requires an order in the last 90 days.
  3. Upstream teams change schemas without knowing who depends on them. Sculley and colleagues (NeurIPS 2015) named "data dependencies" and "undeclared consumers" among the debts specific to machine learning systems. Your AI assistant is, very often, an undeclared consumer.
  4. They try to fix everything at once. A company-wide data model is commissioned, stalls, and ships nothing. The GoCardless lesson is the opposite: start at moments of change.

How Data Architecture for AI Works: The Unlabelled Kitchen

Picture a contract cook starting a shift in a kitchen she has never seen.

She is very good; she has cooked in a hundred kitchens. But here the crates in the walk-in fridge have no labels and there is no recipe book. Nobody tells her that "one portion" of fish is 140 grams in this restaurant. Orders start arriving.

She will not freeze, and that is the point. She will open a crate, decide it is probably cod, guess at a portion, and send out a plate that looks entirely professional. Some plates will be right. Many will be confidently wrong, and the customer cannot tell which from looking.

That is a language model pointed at a raw database: skilled, fast, willing, and new to your kitchen. A column called cust_status is an unlabelled crate, and the guess will arrive in fluent, confident prose.

A more famous cook does not fix this. A better chef in the same unlabelled kitchen makes the same guesses more elegantly: the three-models-tied result in kitchen form.

What fixes it is five things every well-run kitchen already has.

What is a data contract?

A data contract is a written, machine-checked agreement between a data producer and its consumers covering schema, meaning, owner and quality or freshness promises.

In the kitchen, it is the delivery label on each crate: what is inside, the use-by date, and the supplier's phone number, written by the supplier, who actually knows. And the kitchen checks it at the door: a crate with no label, or a label that does not match the contents, is refused before it reaches the fridge.

That check is what "machine-checked" means. A contract that nothing verifies is a wish. At GoCardless the contract was code that built the pipeline, so contract and reality could not drift apart quietly.

What is a semantic layer and why does AI need one?

A semantic layer is one governed place where metrics such as "revenue" or "active customer" are defined once and reused by both reporting and AI. The model needs it because it lets the model read your meaning instead of guessing from column names.

In the kitchen, it is the house recipe book: exactly what "one portion" means here. The word that matters is one. If the sales dashboard and the finance report each carry their own formula for revenue, you do not have a semantic layer. You have two cooks with two private notebooks, and an assistant that will pick one at random.

It does not need to be grand. The 2026 study's semantic layer was one hand-written document of about 4 KB: a page or two, plus the harder work of agreement.

What is data lineage and why does it matter for AI?

Lineage is the recorded path from source to answer, needed to debug a wrong AI answer and to prove provenance to auditors.

In the kitchen, it is the stack of delivery receipts. When a plate comes back, you can trace it to the batch, the crate, the supplier and the day. Without receipts, a bad plate means throwing out the whole fridge or shrugging. In the opening story, lineage is the difference between a two-day hunt and a quick look at a trail someone already wrote down.

What is knowledge source ownership?

Knowledge source ownership means one named person is accountable for each collection of documents an assistant reads: that the current version is the only live version, that it is reviewed by a set date, and that it is retired when it is superseded.

In the kitchen, it is the allergen sheet taped up by the pass. The cook trusts it completely, which is exactly why it needs a name and a date on it. If the recipe changed in spring and the sheet on the wall is last year's, the cook follows the sheet, confidently, and nobody finds out until a customer does.

Think of it as the document version of a data contract, cut down to three promises: an owner, a review date, and a retirement rule (what happens to the old version when a new one is published: moved out of the assistant's reach, not left beside it). Lesson 6 on fixing data quality one use case at a time made a document source ready for a pilot; ownership keeps it ready in production, after its authors have moved on.

What is a knowledge graph?

A knowledge graph stores things and the relationships between them (customer places order, order contains product) so that a model can follow the business's own connections instead of guessing them from column names. That sentence is this course's working explanation, not a formal definition from the benchmark paper.

In the kitchen, it is the map on the wall: which dishes share a sauce, which allergens sit in which base. It answers the "how is this connected to that" questions that a list of ingredients cannot. In a database, those are the questions that need joins, and joins are where a guessing model does the most damage.

Why does the order matter?

Each piece protects the next.

  1. The contract keeps meaning stable upstream, and the owner and review date do the same for documents. If a column can be renamed silently, or an old policy can sit beside the new one, everything built on top is on sand.
  2. The semantic layer states meaning once, and reports and AI read the same page.
  3. Lineage lets you check by following the trail instead of arguing.
  4. The model reads all of it as context. It stops guessing because it has been told.

The knowledge graph comes last on purpose. It pays off when failures cluster around relationships, and only measuring tells you where they cluster. So write the definitions first, score the result, and let the remaining wrong answers tell you whether a relationship map is worth building. The same owned definitions and sources are what Lesson 23 on the data flywheel builds on at Level 5, when people's corrections to these answers start to feed back into the system.

The Workshop: Build an AI Data Foundation Pack for One Use Case

You will run two 90-minute sessions, seven steps in all, and leave with an artifact called the AI Data Foundation Pack: a semantic definitions sheet, a one-page data contract, a knowledge source register and a lineage trace, plus a before and after score. One use case. Not the enterprise.

Who is in the room and what do they bring?

A core of three for both sessions, and two guests who join only for the step that needs them. The hard work here is agreement, and agreement slows with every extra chair.

  • The AI product owner from Lesson 11, who runs the sessions and owns the score.
  • One analytics engineer, who knows how the dashboards really compute things.
  • The business owner of the most-asked metric, often the finance controller, with the authority to sign a definition.
  • Guest for step 3: the nearest thing to an owner of the document collection read most.
  • Guest for steps 4 and 5: the lead engineer of the producing team whose source is about to change.

Inputs: one live or near-live AI use case that answers from structured data and documents, your data inventory from Lesson 2, your golden test set from Lesson 8, and last quarter's management report, the closest thing you have to an official answer key.

Session one: measure, then write the meaning down

  1. Collect 20 real questions and run the baseline. Take them from user logs, policy questions included, not from what the data team finds interesting. Write the correct answer to each from the official report or the current approved document, then ask the system all 20 and record how many it gets right. This is your "before". Without a before, your after is an opinion. Add all 20 to the golden set.
  2. Define and sign the ten business terms. Underline every word in the questions that carries business meaning: active, revenue, region, churned, new. For the ten most used, write the definition, calculation, source table, approving owner and known traps. Expect an argument; the argument is the workshop working. Keep the sheet short (remember the 4 KB document) and get each owner to sign their term by name and date. An unsigned definition is a suggestion.
  3. Register the knowledge sources. List every document collection the assistant reads for this use case. For each, record a named owner, the authoritative version, the next review date and the retirement rule. Then search for other versions of the documents your questions touched; their owner retires the superseded ones in the session. A collection nobody will own goes to the steering committee: assign an owner or take it out of the assistant's reach.

Session two: contract, trace, re-run, publish

  1. Draft one data contract at a moment of change. Choose a source that is about to change (a new stream, a migration, a new field): the GoCardless rule, which works because the producing team already has the file open. On one page record schema, field meanings, owner, consumers, classification, freshness and quality promises, and how a change is announced.
  2. Trace one answer end to end. Take one question and write its lineage: source system, table, transformation, definition used, answer. For a document answer, trace it to the exact file and version. Mark every point where someone says "I think it comes from there". Those marks are your backlog.
  3. Re-run through the release gate, and make every answer show its working. Ask the same 20 questions with the definitions sheet in context and the retired versions gone, through the Lesson 11 release gate like any other change. Record the "after" score and list the answers still wrong, with a guess at why. From now on every answer states which definition and query, or which document and version, it used, so a human can check it in seconds. With roughly a third of answers wrong even in the best published results, this is not a nicety.
  4. Publish, assign and date. Put the sheet, the contract and the register under version control, name the owner of each, put every review date in its owner's calendar, and schedule the next source for its next moment of change.

How many test questions do you need to prove an improvement?

More than 20. Twenty questions is enough to find the traps, not to prove a gain. On 20 questions, the 95% margin of error around a 70% score is roughly plus or minus 20 points, so a move from 9 right to 14 right can sit inside the noise. The 2026 study could call its gain significant because it used 100 questions. Before you report an improvement to the steering committee, grow the set to at least 50 questions and re-run both conditions on the same questions.

What does a filled-in example look like?

Here is an invented one. Every number in it is made up for illustration; no real company is described.

Picture a 600-person wholesale distributor with a sales assistant that answers questions from the order database and from a shared library of pricing and returns policies. The team gathers 20 questions from the assistant's logs, 15 about numbers and 5 about policy. With its current context, the assistant gets 9 of 20 right: 7 of the number questions and 2 of the policy ones.

In step 2 the finance controller and the sales operations lead discover, with some heat, that they have meant different things by "active customer" for two years. They settle it (four of ten terms shown).

TermDefinition agreedCalculation and sourceOwnerTrap found
Active customerAccount with at least one invoiced order in the last 90 daysCount of distinct account IDs in invoices, last 90 daysFinance controllerStatus flag "A" includes dormant accounts
Net revenueInvoiced sales minus credit notes, excluding taxInvoices minus credits, ledger tablesFinance controllerSales dashboard used gross orders
Order dateDate the order was confirmed, not enteredConfirmation timestampSales operations leadTwo date columns with similar names
RegionRegion of the delivery addressDelivery address tableSales operations leadAccount region differs for 1 in 8 accounts

Look at the right-hand column. Every trap is a place where a sensible cook would guess wrong: a status flag of "A" looks exactly like "active". This column is the most valuable thing on the page, and it exists only in people's heads until you ask.

In step 3 the customer service manager joins, and the register turns up two surprises.

CollectionOwnerAuthoritative versionNext reviewRetirement ruleFinding
Returns policiesCustomer service managerReturns policy version 431 March 2027Old version archived out of the index on approval of a new oneVersions 2, 3 and 4 were all live; 2 and 3 retired in the session
Trade price listsNone at the start; commercial director agreed to own itThe current quarter's listFirst working day of each quarterPrevious quarter's list archived on publicationNo owner since the last commercial director left
Product data sheetsProduct management leadSheets in the product wiki30 June 2027Discontinued products archived within five working daysClean

Look at the second row: the assistant quoted prices daily from a folder nobody owned. No model upgrade finds that.

In session two they re-run the questions: 14 of 20 right, 11 of the number questions and 3 of the policy ones. Six are still wrong. Four fail on questions that join returns to orders, so "returns to orders" becomes a candidate for a relationship map: their first evidence-based case for a small knowledge graph. One asks about a loyalty discount no document describes, which is missing content that no indexing fixes; the commercial director will write it. The last goes on the review list.

The lineage trace for "active customers last quarter" breaks once: nobody can say which nightly job fills the invoices table. That goes in the backlog with a name beside it. For the contract they choose the invoices feed, because the billing system upgrade is already planned and the producing team is touching the feed anyway.

What templates do I fill in?

Four pages and a score line. Resist adding fields until you have used these once.

PageFields to completeDone when
1. Semantic definitions sheetTerm; one-sentence definition; calculation; source; owner; approval date; known trapsTen terms, each signed off by one named owner
2. Data contract (one page)Dataset name; producer team and owner; consumers; schema with field meanings; classification; freshness promise; quality checks; change notification rule; versionProducer and at least one consumer have agreed it, and it is in version control
3. Knowledge source registerCollection; owner; authoritative version; next review date; retirement rule; findingEvery collection the use case reads has a named owner and a review date in the owner's calendar
4. Lineage traceQuestion; source system or file; table or version; transformations; definition used; answer; break pointsOne question traced end to end, every break point logged with an owner
Score lineSize of the question set; questions right before; questions right after; answers still wrongBoth runs recorded against the same questions, with the set size stated

What does a filled-in data contract template look like?

Here is the illustrative distributor's contract, to show how little it takes.

Contract fieldIllustrative entry
Dataset nameInvoices feed
Producer team and ownerBilling systems team; billing systems manager
ConsumersFinance reporting; sales dashboard; sales assistant (AI)
Schema with field meaningsAccount ID (the customer account billed); invoice date (date issued); net amount (excluding tax); credit note flag (yes if this row reverses an invoice)
ClassificationCommercially confidential, from the Lesson 2 inventory
Freshness promiseLoaded by 06:00 each working day
Quality checksNo missing account IDs; no duplicate invoice numbers; credit notes always reference an invoice
Change notification ruleAny rename, removal or change of meaning announced to all consumers ten working days ahead
Version1.0, stored in the team repository

Notice the third row. The sales assistant is now a declared consumer, so the billing team can no longer break it without knowing it exists. That line alone is worth the session.

The test of the pack: hand it to someone who was not in the room. If they can say what "active customer" means, who to call about the invoices feed, who owns the returns policy, and where an answer came from, you are done.

Failure Modes and the Metrics That Catch Them

How do I spot this going wrong?

Seven patterns, each with a question to ask in a meeting tomorrow.

Failure modeWhat it looks likeHow to spot it
Buying a better model for a meaning problemA model upgrade is the proposed fix for wrong numbersAsk which definition the wrong answer used. If no one can name it, the model is not your problem
Definitions with no ownerTerms are written down but nobody can change themAsk who can approve a change to "active customer". Two names or none is the finding
The big-bang data modelA programme to model everything before anything shipsCount contracts in production after a quarter. Zero is the signal
Contract as a wiki pageA document nothing checks, so it driftsAsk what happens automatically when a producer breaks it. "Nothing" means it is not a contract
Silent schema changeThe golden-set score drops after an upstream releaseAsk the producing team whether they knew the AI system was a consumer
Ownerless or stale knowledge sourceThe assistant quotes a superseded policy or an old price listTake the folder behind the last wrong policy answer and ask who owns it and when it was reviewed
Treating a better score as doneA gain on 20 questions is reported as proof, and answers reach decision makers uncheckedAsk how many questions the claim rests on, and look for answers sent with no definition, query or source shown

The first and last are the pair a busy executive team falls into together: "the tool is wrong, upgrade it" and "the tool improved, trust it". Both skip looking at what the answer was built from.

Which metrics should I track?

Six, all countable after the workshop.

MetricHow to measure it
Answer accuracy, before and afterCorrect answers divided by questions on a named question set, always reported with the set size
Definition coverageShare of the use case's key terms with an approved definition and a named owner
Contract coverageShare of source datasets feeding the use case that have a contract
Knowledge source freshnessShare of document collections the assistant reads that have a named owner and a review date not yet passed
Unannounced breaking changesUpstream changes per quarter that broke the system without notice, from incident records
Time to traceHours from "this answer is wrong" to "here is the cause", measured on real incidents

In the illustrative distributor, the first report to the steering committee would read: accuracy 9 of 20 before and 14 of 20 after, marked as not yet proven while the set grows to 50; ten of ten terms signed off by a named owner; three knowledge sources registered with owners and review dates, two superseded policies retired; one contract drafted; one lineage break logged.

Report accuracy as a fraction of a named question set, never a bare percentage. "14 of 20 on the sales question set" invites the right follow-up; "70% accurate" invites a decision nobody should make.

Frequently Asked Questions

What is the difference between a data contract and a semantic layer?

A data contract is the producer's promise about a dataset (its schema, meaning, owner and freshness), while a semantic layer defines business terms such as "active customer" once, on top of those datasets, for reports and AI to share. The contract keeps the inputs stable; the semantic layer says what they mean to the business.

Does a semantic layer or knowledge graph really improve text to SQL accuracy?

Yes, in the published benchmarks. Accuracy rose from 16% to 54% with a knowledge graph in a November 2023 data.world study, and from 45.5 to 50.5% up to about 68% with a 4 KB semantic document in an April 2026 study. The 2026 authors appear to be vendor-affiliated, so read that result as indicative, and note that roughly a third or more of answers were still wrong in the best case.

Will a better model fix wrong answers from our data?

Probably not on its own. In the April 2026 study three frontier models scored the same within each condition, and the gain came from the added business context.

Who should own the documents an AI assistant reads?

The business owner of each collection, not the AI team: the customer service manager for returns policies, the commercial director for price lists. That person keeps one live version, reviews it by a set date and retires old versions out of the assistant's reach.

Do we need a knowledge graph before we deploy an AI assistant?

No. Start with signed definitions, one data contract and owned document sources, then measure. Build a small knowledge graph only when the answers still wrong cluster around relationships, such as returns joined to orders.

Key Takeaways

  • Wrong numbers from an AI assistant are usually a meaning problem, not a model problem: in an April 2026 study three frontier models tied, and a 4 KB semantic document added 17 to 23 points.
  • Data architecture for AI is five plain things: a data contract (the crate label), a semantic layer (the recipe book), lineage (the receipts), knowledge source ownership (the dated allergen sheet) and a knowledge graph (the map on the wall).
  • Follow the GoCardless pattern from 2022: small contracts, checked by a machine, introduced one stream at a time at moments of change.
  • Every document collection an assistant reads needs a named owner, a review date and a retirement rule, because contradicting versions confuse retrieval and ownerless folders go stale unnoticed.
  • Run the seven-step workshop on one use case to build an AI Data Foundation Pack: signed definitions, one contract, a knowledge source register, a lineage trace and a before and after score.
  • Twenty questions finds the traps but cannot prove a gain; grow the set to at least 50 before reporting an improvement, and make every answer show its definition, query or source.
  • Track six numbers: accuracy with its set size, definition coverage, contract coverage, knowledge source freshness, unannounced breaking changes and time to trace.