Building Institutional Knowledge
When Renata Alvarez retired after eleven years as the AI program director at a federal grants agency, her going-away party was warm and her exit was a quiet catastrophe. Renata knew which vendor's model quietly degraded every fiscal year-end and why. She knew the unwritten reason the agency had abandoned a facial-recognition pilot in 2023. She knew which data field in the legacy system was unreliable and should never feed a model. None of it was written down. Her successor inherited a folder of slide decks, a dozen production systems, and a phone number Renata stopped answering after a month. Within a year the agency re-ran a failed experiment Renata had already learned from, because the lesson left the building when she did.
This is the problem institutional knowledge solves. In government, where AI programs outlast administrations and staff turnover is constant, the hardest asset to keep is not the technology, it is the accumulated understanding of how the technology behaves in your specific context. It is also, in a federal agency, partly a legal obligation rather than a nice-to-have: the record of why a system exists and what it does is the same record an Inspector General, an auditor, a congressional staffer, a judge or a successor appointee will ask you to produce. This lesson is about capturing that understanding so it survives the people who earned it.
The Knowledge That Walks Out the Door
Organizations confuse two different things. Information is what is already written: the model documentation, the architecture diagrams, the policies. Knowledge is the layer that lives in people's heads: why decisions were made, what was tried and abandoned, which warning signs to watch, who to call when something breaks. Information is easy to keep and surprisingly low-value on its own. Knowledge is what makes a new person effective, and it is exactly what disappears in an exit interview, in a contractor transition, or in the last six weeks of an administration.
For AI systems this is acute. A model is not self-explaining. Knowing that a system exists tells you nothing about why it was tuned the way it was, what biases were found and corrected, what authority it operates under, or what the team learned the third time it failed. Lose that and your successors are not standing on your shoulders, they are starting over. Worse, they are starting over while the system keeps producing decisions that affect the public, because nothing about a knowledge gap stops a model from running.
What Institutional Amnesia Actually Costs
The case record is unusually clear here, and every item in it is a variation on one theme: when an agency cannot reconstruct what it did, why, and with what evidence, oversight and remediation become ruinously expensive. These are not stories about bad algorithms. They are stories about institutions that could not answer questions about their own past.
- IRS ID.me. The 2022 identity-proofing crisis traced in part to a memory gap between identity-proofing pilot discussions in the 2017 to 2020 period and the 2021 to 2022 rollout. Fairness concerns raised earlier did not reach the deployment decision with the force they needed. Remediation included publishing fairness reports and strengthening Chief AI Officer processes.
- Michigan MIDAS. The system continued generating false fraud accusations for nearly two years despite internal concerns raised early. Those concerns had no durable institutional channel through which to reach state leadership. The state later adopted statutory limits on automated adjudication.
- Houston HISD EVAAS. In Houston Federation of Teachers v. Houston ISD (S.D. Tex. 2017), the district could not reconstruct how specific teachers had been scored by the model. The court questioned whether due process could be provided when the institution itself did not know.
- K.W. v. Armstrong (D. Idaho). A Medicaid disability-benefits allocation algorithm the state could not fully explain, because the knowledge behind it had not been captured. The court ordered disclosure and reform.
- SyRI. The Dutch risk-scoring system was struck down in 2020 in part because the government had not preserved and published the evidence that would have demonstrated necessity and proportionality.
- The Dutch childcare benefits scandal (toeslagenaffaire). Made far worse by institutional reluctance to surface internal warnings across ministries, leading to the resignation of the Rutte III cabinet in January 2021 and multiple parliamentary commissions.
- The UK Post Office Horizon scandal. When institutional memory is weak and the documentation is controlled by the vendor, the public can be harmed on an enormous scale before the record can be reconstructed at all.
Read those together and a pattern emerges that should change how you budget. In several of these cases the warning existed inside the institution before the harm did. What was missing was not insight, it was a durable channel that carried the insight forward in time. Institutional knowledge is the infrastructure that turns a concern raised early by one person into a constraint the institution still honors long after that person has gone.
Three Kinds of Knowledge Worth Capturing
You cannot write down everything, and trying produces documentation no one reads. Focus on the three categories that hurt most when lost, and accept that a thin, maintained record of these three beats an exhaustive record of everything that nobody updates.
- Decision rationale. Why you chose this model, this vendor, this threshold, and what you rejected. This is what prevents your successors from re-litigating settled questions or repeating rejected ones.
- Failure memory. What was tried and did not work, and why. Renata's abandoned pilot is the canonical example. Failure memory is the single highest-value and most-neglected category.
- Operational tacit knowledge. The quirks: the unreliable data field, the seasonal model drift, the workaround everyone knows but no one wrote down. This is what makes the difference between running a system and merely owning it.
The Records You Already Owe
Before you design anything, it is worth knowing that a large part of this is already law, and that treating AI artifacts as exempt from records obligations is a mistake agencies make routinely. The statutory backbone is the Federal Records Act (44 USC 31), the Presidential and Federal Records Act Amendments of 2014 that closed electronic records loopholes, the records schedules issued by the National Archives and Records Administration, including General Records Schedule 3.2 covering information technology records, the Paperwork Reduction Act (44 USC 35), and the Freedom of Information Act (5 USC 552).
Two consequences follow that teams consistently miss. First, records are created by whatever documents a decision, not by whatever sits in the official repository: email, chat, code repositories, notebooks and model weights all create records when they document decisions about a system. Second, retention has to match downstream exposure. For rights-impacting AI, the source guidance is that retention should run long enough to support litigation, audit review, and remedy for affected individuals. Your agency's records officer sets the actual schedule. Never assume a default, and never set one yourself.
The OPEN Government Data Act of 2019 added a related obligation by creating Chief Data Officers in each agency and requiring agencies to maintain enterprise data inventories. That inventory is a memory asset even though it was not designed as one, because it is the only place many agencies can answer the question of what data a model was permitted to consume in the first place.
What the Governing Frameworks Assume About Memory
Office of Management and Budget Memorandum M-24-10, issued in 2024, requires agencies to publish an annual AI use case inventory, designate a Chief AI Officer, complete AI impact assessments for rights-impacting and safety-impacting AI, and apply minimum risk-management practices that explicitly include documentation. Read carefully, the memo is a memory instrument as much as a governance one. Almost every obligation it imposes assumes the agency can recall and evidence something it did earlier, and an agency that cannot recall cannot comply.
The NIST AI Risk Management Framework 1.0, published in January 2023, is voluntary and non-binding, but it places documentation at the center of every function: GOVERN 1 on policies and documentation, GOVERN 5 on decision records, MAP 1 on context including organizational history, MEASURE 4 on feedback integration, and MANAGE 4 on continuous monitoring, learning and incident response. Executive Order 14110, issued in October 2023, further required documentation of testing, red-teaming and dual-use foundation model reporting; treat it as a historical instrument and confirm its current status before relying on it.
Two international standards point the same way. ISO/IEC 42001 for AI management systems and ISO/IEC 23894 for AI risk management guidance both institutionalize documentation and continuous improvement. The GAO Green Book supplies the complementary discipline of communicating risk and performance upward and sideways, so that decisions are actually informed by what the agency already knows. That last clause is the one that matters: a record nobody routes to a decision-maker has no governance value at all.
The Living Dossier
The organizing artifact is a dossier per system, not a library of unrelated documents. Every rights-impacting or safety-impacting AI system gets one, and it is a living file rather than a launch deliverable. The source practice is that the dossier is updated quarterly and on material change, which is the detail that separates a dossier from a binder. A document written once and never revisited answers the question "how do you know?" very badly, no matter how thorough it was on the day it was written.
| Dossier element | What it answers |
|---|---|
| Statutory and regulatory authority | Under what authority does this system operate at all |
| Intended use, scope, user population | What was this built to do, and for whom |
| Data sources with provenance, license, and SORN references | Where the data came from and what permits its use |
| Model architecture and version lineage | Which version produced a given decision |
| Training and evaluation history, including fairness metrics tied to NIST AI RMF MEASURE 2.11 | What was tested, when, and with what result |
| Known limitations and failure modes | What we already know will go wrong |
| Waivers | What was excepted, by whom, and on what basis |
| Incidents with after-action outcomes | What went wrong and what changed because of it |
| Retirement criteria | What would make us turn this off |
| Responsible role identifiers | Who owns this, by role rather than by name alone |
The last row deserves emphasis. Recording the role rather than only the individual is what lets the dossier survive the individual, which is the entire point of the exercise. Renata's agency had most of the technical facts somewhere. What it did not have was any artifact that named a continuing owner responsible for keeping those facts true.
Capturing Knowledge Without Killing It With Bureaucracy
The instinct is to demand a giant document. That fails twice: people resist writing it, and no one reads it. Knowledge capture has to be lightweight, continuous, and built into how work already happens. Three habits carry most of the load, and each one populates part of the dossier as a by-product rather than as a separate compliance chore.
Decision records, written at the moment
The most reliable technique is a one-page decision record written when a decision is made, not reconstructed years later. Each one answers four questions: What did we decide? What were the alternatives? Why this one? What would make us revisit it? A decision is freshest the day it is made; capturing it then costs minutes and saves your successor months. Over time these records become the narrative of why your AI program looks the way it does, and they map directly onto the framework expectation that an agency can produce decision records on request.
A living failure log and structured after-action reviews
Keep a standing, blameless record of experiments that did not pan out: what was tried, what happened, and what you concluded. The blameless part is essential. If the failure log reads like a list of mistakes people will be judged for, it stays empty. For incidents rather than experiments, the discipline is a structured after-action review captured against a standard template, covering event description, root cause analysis, corrective actions, and the policy updates that followed. Share it inside the agency, and publish it where publication is appropriate.
System runbooks that include the quirks
Standard runbooks describe how to operate a system. Add the section most of them omit: the known quirks, the warning signs, the "if you see X, it means Y" knowledge. This is where the unreliable data field and the year-end drift get written down. It is the closest thing to bottling what an experienced operator knows, and it is usually the fastest of the three artifacts to produce, because operators already carry the content in their heads and simply have never been asked.
A Lessons-Learned Repository, and Why It Should Cross Agency Lines
A single agency generates too few incidents to learn efficiently from its own experience alone. A shared repository, internal first and cross-agency where the material allows, converts a rare local event into a common warning. Entries carry the event description, the root cause analysis, the corrective actions and the resulting policy updates, anonymized where necessary. The interagency Chief AI Officer Council convened under M-24-10 is the natural place to compile patterns across agencies and flag emerging issues before each agency discovers them independently.
The direction of travel matters. Lessons from an identity-proofing failure at one revenue agency are directly relevant to every other agency that does identity proofing at scale, and the source guidance is explicit that they should be shared to those components rather than filed locally. The obstacle is almost never technical. It is that publishing a failure feels like creating discoverable evidence of incompetence, which is precisely the instinct the blameless framing exists to defeat.
Succession and Workforce Knowledge Transfer
Knowledge transfer is far cheaper when it overlaps with the person who has the knowledge. Renata's agency lost everything because transfer started the week she left. Three practices fix this, and none of them are expensive relative to what a single re-run failed experiment costs.
- Document continuously, not at exit. If the only knowledge transfer happens in someone's final two weeks, you have already lost most of it. The records above should accumulate throughout a tenure.
- Never single-thread critical knowledge. Where one person is the only one who understands a system, that is a risk to retire deliberately: pair them with someone, rotate responsibilities, force the knowledge to spread.
- Build overlap into transitions. Arrange a handover period where the departing and arriving people work side by side on real problems. Shadowing transfers tacit knowledge that no document captures.
Structured onboarding is the mirror image of a structured exit, and for a new Chief AI Officer it should include briefings with prior officeholders, whether political or career, a walk through the dossiers, the history of Inspector General and GAO findings, the congressional correspondence, and the budget context. Rotational mechanisms carry knowledge in the other direction. Assignments under the Intergovernmental Personnel Act, Title 5 details, academic affiliations and sabbatical exchanges with research organizations such as MITRE or RAND are usually described as talent pipelines, but they are knowledge vehicles too, moving practice between institutions in both directions.
The Use Case Inventory as a Memory Tool
M-24-10 requires the inventory. The move that turns a compliance artifact into a memory system is to make each inventory entry the gateway to everything else: a link to the public-facing summary, to the internal dossier, to the most recent AI impact assessment, to the current evaluation results, and to the incident count. Done that way, the annual inventory obligation maintains itself, because the inventory is the index everyone already uses rather than a spreadsheet rebuilt each year from scratch.
Inventories rot, and the rot is quiet. Three controls hold it back: quarterly certification by each named system owner, automated checks that flag any inventory entry with no corresponding dossier, and sample audits by the Chief AI Officer's own office. Note what these controls have in common. Each one forces a person to look at a record on a schedule. Documentation nobody is ever required to open degrades into fiction at roughly the speed the underlying system changes, and it degrades invisibly, which is worse than being absent.
Public Accountability as Memory Discipline
The uncomfortable truth is that most agencies improve their memory only when forced. FOIA readiness forces an agency to know what it actually has. Congressional engagement builds memory by requiring the agency to reconstruct and explain decisions out loud. Civil society engagement through comment periods and advisory bodies, academic partnerships under cooperative research and development agreements and cooperative agreements, and GAO audits all act as external memory prompts that no internal process reliably substitutes for.
The variable is posture. Agencies that treat these interactions as adversarial lose memory, because everything becomes a document to be minimized and a question to be narrowed. Agencies that treat them as learning opportunities gain memory, because each request becomes an occasion to consolidate a record that was scattered. This is one of the few governance choices where the defensive strategy and the effective strategy point in opposite directions, and leaders should say so plainly to their teams.
The Transition Playbook
Administration transitions are the sharpest single point of loss, and the source failure mode is specific: context is lost in the last six weeks. The countermeasure is a standing AI posture package that is maintained continuously and handed over rather than assembled under pressure. It should cover active rights-impacting systems, known risks and residual risks, pending waivers, open Inspector General and GAO correspondence, litigation posture, budget state, a personnel continuity map, external partnership commitments, and a priority stack.
Cross-agency councils and communities of practice are the natural keepers of continuity that no single agency can hold on its own, because they persist across the transitions that reset individual agencies. Build the relationship before you need it. An interagency peer who remembers why your agency abandoned an approach in a prior administration is, functionally, a piece of your institutional memory stored outside your building.
The Failure Modes to Watch For
The source names seven recurring failure modes, and they are worth reading as a diagnostic rather than a list. Knowledge lives in email and chat rather than in durable records. Dossiers exist for pilots but not for legacy systems. The inventory lists systems but not evidence. After-action reviews are either not done or not shared internally. Transitions lose context in the last six weeks of an administration. Inspector General findings are addressed narrowly rather than systemically. And no owner is accountable for long-horizon knowledge health, which is the failure that produces all the others.
Organizational Memory for AI Specifically
Pull these pieces together into one accessible place: a knowledge base that a new program director can read in a week and come away knowing the real history, not the sanitized version. The aim is that someone joining your AI program can answer, from the record alone, three questions. Why do our systems look the way they do? What have we already tried and learned? What should I be watching for? An agency that can answer those from its records rather than from one retiring expert's memory has built genuine organizational memory.
Had Renata's agency kept even a thin version of this, decision records, a blameless failure log, runbook quirks, and a dossier with a named continuing owner, her successor would have opened the failure log, found the 2023 pilot, read why it was abandoned, and saved a year. The technology did not fail them. Their memory did, and the memory was the cheaper thing to build.
An institutional-knowledge starter kit
- A living dossier per rights-impacting system. Authority, intended use, data provenance, model lineage, evaluation history, limitations, waivers, incidents, retirement criteria, responsible roles, refreshed on a schedule and on material change.
- Decision records. One page per major choice: decision, alternatives, rationale, revisit triggers, written at the moment.
- Blameless failure log and after-action template. A standing record of what was tried and did not work, plus a standard structure for incident reviews that ends in corrective actions and policy updates.
- Runbooks with quirks. Operating instructions plus the known warning signs and "if X then Y" tacit knowledge.
- An inventory that indexes the evidence. Each entry linked to dossier, impact assessment, evaluation results and incident count, with owner certification on a cycle.
- A records schedule agreed with your records officer. Covering AI artifacts explicitly, including the chat, code and notebook records that document decisions.
- Single-threading register. A list of systems only one person understands, with a plan to spread that knowledge.
- Overlap-based handovers and a standing transition package. Side-by-side transitions rather than exit-week document dumps, and an AI posture package that is maintained rather than assembled.
Anti-Patterns
- Documentation mistaken for memory. The most common failure in this domain is an agency with a full shelf and no recall. A document that exists but was never revisited, never routed to a decision-maker and owned by nobody is not institutional knowledge. The control is not the artifact, it is a living artifact with a named owner and a review cadence. If you cannot say who is responsible for keeping a document true, assume it is already false.
- The exit interview as a knowledge transfer plan. Scheduling knowledge capture for someone's final two weeks guarantees you capture the shallowest layer. By the time a departure is announced, the person is fielding goodbyes, closing out work and, often, already mentally elsewhere. Anything that only exists in their head at that point is most likely lost. Continuous capture is not a nicer version of an exit interview; it is the only version that works.
- Compliance completion treated as evidence of memory. A published inventory, a filed impact assessment and a completed minimum-practices checklist tell you an agency did the paperwork. They do not tell you the agency could reconstruct, under oath or under audit, how a specific decision was reached about a specific person. Test the second question directly by picking one production system and asking for the evidence trail, and do it before an auditor does.
- Treating AI artifacts as outside records obligations. Teams routinely assume that notebooks, chat threads, model weights and evaluation runs are working material rather than records. Where they document decisions, they create records, and the schedule that governs them is set by your records officer, not by the project team's convenience. Guessing a retention period, or quietly deleting on a project-cleanup cadence, converts a documentation problem into a legal one.
- Blame-shaped failure logging. A failure log that is used in performance conversations will be empty within a quarter, and its emptiness will be read by leadership as evidence that nothing is going wrong. This is the most expensive silence in the domain, because the case record shows the warning usually existed inside the institution before the harm did. Protect the blameless framing explicitly and visibly, or do not bother building the log.
- Fixing the finding rather than the class. Addressing an Inspector General finding narrowly, on the specific system named, leaves every sibling system carrying the same defect. Ask what class of problem the finding represents and sweep for it, then record the sweep. The narrow fix is faster and reliably produces the same finding again two systems later.
Practice Prompts
- Run a reconstruction drill. Choose one production AI system and one decision it made about a member of the public. Try to reconstruct, from records alone, the model version in use, the data sources it consumed, the evaluation evidence current at that time, and the authority under which it operated. Note every gap and who would have had to be interviewed to fill it. That list of names is your single-threading register.
- Draft a dossier for your highest-stakes system. Use the element list above. Fill what you can from existing records, mark the rest as unknown rather than guessing, and take the unknowns to whoever is most likely to still remember. Then set a review cadence and name a responsible role, not just a person.
- Write three retrospective decision records. Pick three consequential choices your program made in the past two years and write the one-page record for each: decision, alternatives, rationale, revisit triggers. Notice how much harder the third one is than the first. That difficulty is the argument for writing them at the moment.
- Audit the inventory against the evidence. Take your published use case inventory and check, entry by entry, whether a dossier and current evaluation results actually exist behind each one. Report the percentage that pass as a baseline, then set a certification cycle and re-measure.
- Build the transition package now. Draft the AI posture package as if a transition were six weeks away: active rights-impacting systems, risks and residuals, pending waivers, open oversight correspondence, litigation posture, budget state, personnel continuity, partnerships, priority stack. Keep it current from here rather than rebuilding it under deadline.
- Sit with your records officer. Bring a list of the artifact types your AI program actually produces, including chat threads, notebooks, code repositories, evaluation outputs and model weights, and ask which schedules apply. Take their answer as the authority. Write down what you learn and attach it to the dossier template.
Reflection
Think about the person on your team who would be hardest to replace, and be honest about why. Is it their skill, which you could hire again, or is it what they know about your particular systems, which you could not? Now ask what would actually happen if they gave notice tomorrow. Which questions would become unanswerable? Which decisions would quietly get re-made from scratch? Write down the three specific things they know that exist nowhere else, then decide which of the three artifacts in this lesson would hold each one.
Then turn the question outward. If an Inspector General asked your program to explain, with evidence, why one of your models uses the threshold it uses, how long would that take and how many people would have to be found? If the honest answer is that it depends on whether a particular person is still reachable, you have identified your most urgent piece of work, and it is not a technology project.
Glossary
- Institutional knowledge. The accumulated understanding of why systems exist, what was tried, what was rejected and what to watch for, as distinct from the documentation that describes what a system is.
- Dossier. A living per-system file holding authority, intended use, data provenance, model lineage, evaluation history, limitations, waivers, incidents, retirement criteria and responsible roles, updated on a cadence and on material change.
- Decision record. A one-page account written at the moment of a decision, covering what was decided, what alternatives existed, why this option was chosen and what would trigger revisiting it.
- After-action review. A structured post-incident review against a standard template covering event description, root cause, corrective actions and resulting policy updates, shared internally and published where appropriate.
- Records schedule. The retention and disposition rules governing a category of records, issued by the National Archives and Records Administration or adopted agency-specifically, and set by your agency's records officer rather than by a project team.
- AI use case inventory. The annual public listing of an agency's AI use cases required by OMB Memorandum M-24-10, which becomes a memory tool when each entry indexes the evidence behind it.
- Single-threading. The condition in which exactly one person understands a system well enough to operate or explain it, treated as a risk to be retired deliberately rather than a staffing fact.
- Transition package. A maintained handover artifact describing AI posture at an administration change: active systems, risks, waivers, oversight correspondence, litigation, budget, personnel continuity, partnerships and priorities.
Related Lessons
- AI Use Case Inventory and Documentation (OMB M-24-10) covers the inventory obligation itself in depth, which is the artifact this lesson turns into an index of institutional memory.
- Oversight Mechanisms: IG, GAO, Congress explains who will ask your agency to reconstruct its decisions and what evidence each of them expects to see.
- AI Incident Documentation and Response goes deeper on the incident record and the after-action review that feeds the failure log described here.
- Accountability Frameworks for AI Failures takes up what happens after a failure is documented, including how corrective action is assigned and tracked.
- When Government AI Goes Wrong works through the case record in more detail, including several of the cases summarized in this lesson.
- Workforce Planning for AI addresses the staffing side of single-threading, rotation and succession that this lesson treats only as a knowledge problem.
Closing
Institutional knowledge is unglamorous work that competes badly for attention against launching things. It has no demo. Its benefit shows up as an absence: the crisis that did not happen, the experiment that was not repeated, the audit that was answered in a week rather than a quarter. That invisibility is exactly why it needs an owner with authority rather than goodwill, and why the framework and records obligations covered here are useful to you politically as well as legally. They convert a good idea into an obligation with a name on it.
Start smaller than feels adequate. One dossier for your highest-stakes system, one decision record written this week, one blameless failure log with the first entry contributed by whoever is most senior. Seniority matters for that first entry, because the norm is set by who goes first. Every case in the record above involved an institution that could not answer questions about its own past. Being able to answer is not primarily a technical capability. It is a habit, maintained by people who will not be there to benefit from it.
Key Takeaways
- Knowledge, not information, walks out the door. Agencies keep documentation reasonably well and lose the rationale, failure memory and tacit operational quirks that actually make a successor effective.
- The case record shows amnesia is expensive, not embarrassing. ID.me, MIDAS, EVAAS, K.W. v. Armstrong, SyRI, toeslagenaffaire and Horizon all turn on an institution that could not reconstruct what it did, why, and with what evidence.
- Much of this is already a records obligation. The Federal Records Act, NARA schedules including GRS 3.2, the Paperwork Reduction Act and FOIA govern AI artifacts too, and chat, notebooks and model weights create records when they document decisions.
- M-24-10 and the NIST AI RMF both assume recall. The inventory, impact assessments, decision records and monitoring functions all presuppose an agency that can produce evidence of its own past choices.
- The dossier is the organizing artifact. One living file per rights-impacting system, updated on a cadence and on material change, with a responsible role recorded rather than only a name.
- A document nobody maintains is not memory. The control is a living artifact with a named owner and a review cadence; an unread, unrevised document answers "how do you know?" badly no matter how complete it looked at launch.
- Capture at the moment, blamelessly, and lightly. One-page decision records, a blameless failure log, and runbooks that include the quirks beat any document large enough to feel comprehensive.
- Design for the transition you know is coming. Overlap-based handovers, a single-threading register and a maintained AI posture package prevent the last-six-weeks loss that the failure record keeps naming.
Frequently Asked Questions
Is institutional knowledge just records management with a new name? No, though the two overlap and the records obligations are real. Records management asks what you must retain and for how long. Institutional knowledge asks whether a successor can act correctly using what was retained. An agency can be fully compliant with its records schedules and still be unable to explain why a model uses the threshold it uses, because the rationale was never a record in the first place. Build both, and treat the records officer as a partner rather than a constraint.
How much of this should be public? The use case inventory is published under M-24-10, and after-action reviews should be published where publication is appropriate. Beyond that, the calculus is your agency's, involving privacy, security and legal considerations that this lesson cannot decide for you. The useful principle is that FOIA readiness improves memory regardless of what you ultimately release, because preparing to answer forces you to know what you hold. Decide the release question with counsel, and the knowing question on your own.
Who should own institutional knowledge for an AI program? In a federal agency the Chief AI Officer is the natural convener, working with the records officer, the Chief Data Officer, the privacy official, the security officer and general counsel. The specific answer matters less than that the answer exists and is written down. The seventh failure mode in the list above is that nobody is accountable for long-horizon knowledge health, and it is the one that generates the other six.
We are a state or local agency, not federal. Does any of this apply? The mechanisms transfer directly: dossiers, decision records, blameless failure logs, runbook quirks, succession overlap and a transition package are not federal inventions. The specific instruments do not transfer. Your state has its own public records law, its own retention schedules and its own procurement and oversight bodies, and the MIDAS case is a state case. Map each practice here to your own jurisdiction's equivalent, and confirm the retention rules with whoever holds that authority locally.
Our program is small and everything is in two people's heads. Where do we start? Start with the single-threading register, because naming the exposure is faster than fixing it and it makes the case for everything else. Then write runbook quirks first rather than decision records, since the operators can produce them from memory in an afternoon. Decision records only pay off going forward, so begin them at the next real decision rather than trying to reconstruct the backlog, which is slow and produces the least reliable records you will ever hold.
Skill.re