Gershom Vasquez-Lindqvist was appointed the first Chief AI Officer (CAIO) of a federal grant-making agency on a Monday, and by Wednesday he understood that his title described a job nobody had defined. The memo announcing his role ran four sentences. There was no AI governance board, no inventory of AI systems, no clear line between his authority and that of the Chief Information Officer, the Senior Agency Official for Privacy, or the agency's civil rights office. When a program director asked him, point-blank, "So if my team wants to use an AI tool to score grant applications, who actually says yes?", Gershom did not have an answer. The honest version was: nobody yet. Building the structure that could answer that question was now his job.

This lesson is about that structure: the roles, bodies, and decision pathways that make AI governance real inside a public-sector organization. A policy without a governance structure is a wish. The structure is what turns principles into decisions that someone is accountable for. By the end you will be able to judge where your agency's governance sits today, design a structure sized to your organization rather than to someone else's, allocate decision rights explicitly, and recognize the failure modes that leave governance existing on an org chart without ever changing an outcome.

Why Structure Comes Before Policy

Agencies often try to write an AI policy before deciding who governs AI. This is backwards. A policy that says "high-risk AI systems require additional review" is meaningless until someone is named to define "high-risk," someone is empowered to require the review, and someone is accountable when the review is skipped. Gershom's first realization was that he could not write a usable policy until he had built the structure that would carry it out.

The symptoms of a missing structure are specific and recognizable. A team wonders whether it should escalate a fairness concern to someone, finds no formal escalation path, and so nothing happens. A committee tries to decide the approval threshold for deploying a new system, finds no documented decision right, and guesses. A governance officer wants to require minimum practices but has no authority to impose them across departments. In each case the intent was good. What was missing was a structure that made the good intent operative.

For federal agencies, this is not optional. OMB Memorandum M-24-10, issued in 2024, requires covered agencies to designate a Chief AI Officer, and many to convene an AI Governance Board, with defined responsibilities for managing AI risk. The memo gives Gershom both his mandate and his blueprint. Proper structure does not guarantee good governance, but it creates the conditions under which good governance can happen: it clarifies who decides what, ensures decisions are documented, creates a mechanism for escalation, and allocates responsibility in a form people can understand and accept.

Reading Your Starting Point: Governance Maturity

Before designing governance, understand where you already are. Agencies sit at different levels of governance maturity, and a structure that fits an organization with nothing in place will not fit one that already documents its decisions. The four levels below describe a progression from ad hoc practice to governance that is itself managed and improved. Gershom began by walking his own agency down this list honestly, which produced an uncomfortable but useful conclusion: on the day he arrived, his agency was at Level 1 in everything but title.

Treat this ladder as a self-description, not a score anyone confers. No body certifies an agency at Level 3, and nothing about occupying a level is measured from outside. What the levels give you is a shared vocabulary for an internal argument about where you actually are, which is why the assessment is only worth as much as the evidence you bring to it. An agency that claims Level 3 should be able to produce the documented decisions; if it cannot produce them, it is asserting Level 3 and operating at Level 1. Moving up a level is generally a matter of quarters rather than weeks, and planning estimates in the range of twelve to eighteen months per level are common.

The Core Roles

The Chief AI Officer

The CAIO holds agency-wide responsibility for AI: coordinating the agency's AI activities, setting strategy, advising and reporting to leadership, overseeing risk management, and serving as the focal point for AI governance. The role carries real decision authority. What it is not is a gatekeeper who personally approves every model. Gershom's job was not to review every AI tool himself, that would not scale, but to ensure a system existed that reviewed the right ones at the right depth. In smaller organizations the same accountability often sits with a Chief Data Officer instead.

The AI Governance Board

The governance board is where cross-functional decisions get made. Its power comes from who sits on it. An effective board includes the people whose domains AI touches: the CIO (technology and security), the Senior Agency Official for Privacy (Privacy Act and data), the civil rights or equity office (discrimination and fairness), legal counsel (statutory authority and liability), procurement (how AI enters the building), and program leadership (the mission the AI serves). Gershom chaired the board; the board, not Gershom alone, made consequential calls. That distinction protected both the decisions and the decider.

The Supporting Governance Roles

Around the board sit the roles that make governance a daily operation rather than a monthly meeting. An AI Governance Officer runs day-to-day operations: scheduling reviews, tracking documentation, chasing follow-through. A Data Owner per dataset answers for data quality, provenance, access control, and retention. A Fairness Lead evaluates fairness implications and oversees fairness assessments. A Privacy Officer evaluates privacy implications and compliance with privacy requirements. A Security Officer evaluates security and adversarial robustness. A Domain Expert validates whether an AI approach is appropriate to the problem at all.

The Program and System Owners

Every AI system needs a named owner, sometimes titled Model Owner: the official accountable for that specific system's performance, updates, monitoring, risk, and compliance. When the grant-scoring tool drifts or produces a disparate impact, the system owner is the person who answers for it. Governance fails when systems are "owned" by a contract or a team rather than a person. Gershom's rule: no AI system enters production without a named human owner of record. In a small agency one person may hold several of these titles at once; in a large one they are separate positions with specialized staff.

The Workforce

Governance is not only senior roles. Every employee using AI is part of the structure, responsible for using approved tools appropriately and raising concerns. A board that meets monthly cannot catch what a clerk sees daily. The structure must include a path for frontline concerns to reach the people who can act on them, and that path has to be one a frontline employee can actually use without a sponsor, a form number they have never seen, or a reason to expect the report will be held against them.

Sizing the Structure to the Agency

The right governance structure depends on organization size. A ten-person team needs different governance than a ten-thousand-person agency, and importing a large agency's committee lattice into a small one produces the bottleneck failure described later in this lesson. The three profiles below scale the same functions rather than adding new ones: someone accountable, a body that decides, and named owners for data and systems.

Small Agency

Roughly 10 to 100 people running 1 to 10 AI systems. Governance is lightweight. Often a single person, the Chief AI Officer or Chief Data Officer at 1 to 2 full-time equivalents, oversees AI with support from a steering committee of 8 to 12 people drawn from leadership, operations, compliance, and technology, meeting monthly. Working groups form ad hoc for specific systems. The steering committee approves high-risk systems, the Chief AI Officer approves medium-risk systems, and team leads approve low-risk ones. Expect one to two months to define governance, two to four weeks to train staff, ongoing application to new systems, and six to twelve months to retrofit existing ones.

Medium Agency

Roughly 100 to 1,000 people running 10 to 50 AI systems. Governance is more formalized, with a Chief AI Officer at 2 to 3 full-time equivalents leading an AI Governance Committee of 12 to 15 cross-functional leaders, supported by a Technical AI Committee of engineers and data scientists, a Fairness and Ethics Committee of subject matter experts, a Data Governance Committee of data managers and privacy officers, and an Incident Response Team formed as needed. The steering committee makes strategic decisions quarterly; the governance committee reviews every new system before deployment; risk classification determines review intensity; escalation paths are explicit. Design takes two to three months, hiring or assigning staff one to two months, building processes and tools three to six months, extensive training two to three months, and full implementation six to eighteen months.

Large Agency

1,000 or more people running 50 or more AI systems. Governance is specialized: the Chief AI Officer is a senior executive with an office of 5 to 10 full-time equivalents, multiple committees cover steering, policy, technical review, fairness, data, and compliance, and dedicated governance and compliance staff run to 20 to 40 full-time equivalents. Distributed organizations add regional governance structures, and the whole apparatus integrates with enterprise risk management. Decision rights are written down, review thresholds are risk-based, escalation protocols exist, and the structure is reviewed on a schedule. Design runs three to six months, supporting structures and processes six to twelve months, hiring dedicated staff three to six months, training the extended organization six to twelve months, and implementation with optimization twelve to twenty-four months.

Decision Rights: Who Can Decide What

Clear decision rights are the substance of a governance structure. The question is not only who approves a deployment, but what happens when data changes, when scope expands, and when measured risk exceeds the tolerance the agency set for itself. Writing this down converts a standing argument into a lookup. The framework below is an example to adapt, not a standard to adopt: the risk tiers must map to your own classification scheme, and the approving bodies must be ones that exist in your agency.

Decision Pathways: Answering "Who Says Yes?"

The program director's question, who actually approves an AI use case, is the test of whether a governance structure works. Gershom designed a tiered intake process so the answer was always clear and proportionate to risk:

1. Intake and triage. Any proposed AI use case is registered through a single intake form that captures the task, the data, the population affected, and the vendor if any. Intake is the route by which governance learns a use case exists; anything that never reaches it is governed by nobody.
2. Risk classification. The use case is classified as rights-impacting, safety-impacting, or neither, using the M-24-10 definitions. The classification, not the enthusiasm of the sponsor, determines the path.
3. Proportionate review. Low-risk, low-impact tools get a lightweight check and a fast yes. Rights- or safety-impacting systems go to the full governance board, trigger an AI impact assessment, and must satisfy the minimum risk-management practices before deployment.
4. Named approval and documentation. Every decision is recorded: what was approved, by whom, on what conditions, and who owns the result. The record is the artifact an inspector general or oversight review will ask for.

With this in place, Gershom could finally answer the program director: a grant-scoring tool is rights-impacting because it affects who receives federal funds; it therefore goes to the full board, requires an impact assessment, needs a named system owner, and cannot launch until minimum practices are met. The answer was no longer "nobody." It was a clear, defensible path, and because it was written down, the next program director got the same answer without asking Gershom.

Two Implementations, End to End

A 50-person agency with three low-risk systems

The starting structure was a Chief Technology Officer with minimal AI focus, a small data team, and no formal governance. The agency did not hire. It designated the CTO as Chief AI Officer within the existing role, created an AI Steering Committee of the CTO, HR, Operations, Compliance, and two external advisors, set the committee to meet monthly, made the data lead the Data Owner for all datasets, and designated technical staff as Model Owners for their systems. Processes were deliberately small: monthly steering reviews of 30 to 60 minutes, annual impact assessments for all systems, quarterly fairness checks, and simple documentation standards.

The timeline ran one month to design governance, two weeks to brief staff and stakeholders, two months to implement, and three months to the first comprehensive review. The cost was roughly 50 hours of CTO time plus 30 hours from other staff, with minimal additional budget. The result was clear governance at low overhead: the systems were better understood than before, and the risks were being monitored rather than assumed. Nothing in this profile required a new headcount, which is the point. Small agencies fail at governance far more often from importing a heavy structure than from lacking one.

A 5,000-person federal agency with 80 AI systems

Here the starting structure had a Chief Information Officer and a Chief Privacy Officer, limited governance specific to AI, and inconsistent practices across divisions. The agency hired a Chief AI Officer as a new executive role reporting to the CIO and stood up an AI Governance Office of two managers and three analysts. It then created five standing bodies: an AI Steering Committee of 14 leaders from across the agency, a Technical AI Committee of 12 technical experts, a Fairness and Ethics Board of 12 subject matter experts, a Data Governance Committee of 8 data professionals, and an Incident Response Team of 5 senior staff.

Every system was put through risk assessment and classification, with high-risk systems taking a comprehensive impact assessment. The governance committee met monthly, the technical committee biweekly, and the fairness board conducted quarterly audits. Data governance set standards, monitoring dashboards covered all systems, and incident tracking and response procedures were written. The timeline was four months to design, six months to build supporting infrastructure, three months to hire the CAIO and staff, three months to train the organization, and 18 months to implement across that period. Cost ran $1 to $2 million in Year 1 for new hires, infrastructure, and training, and $0.5 to $1 million annually thereafter.

The outcome was visibility. The organization could see its AI, risks were managed proactively rather than discovered, and the culture moved toward responsible adoption. Note what the two cases share despite the difference in scale: someone accountable by name, a body with the authority to say no, owners attached to data and to systems, and a written record of what was decided. Everything else is sizing.

Making the Structure Real, Not Decorative

The most common failure is a governance structure that exists on paper but never bites. Gershom guarded against this with a few deliberate design choices. The board had a standing agenda and a published cadence, so it could not quietly stop meeting. Every AI system was entered into a central inventory tied to the IT asset registry, so "we forgot that one existed" became impossible. The intake form was made the only lawful route to procure or deploy AI, enforced through the procurement office. And the board's decisions were documented in a form that could survive his own departure.

That last choice matters more than it sounds. A structure that depends on one person is not a structure; it is a bottleneck waiting to become a vacancy. It is also worth being clear about what the procurement enforcement does and does not reach. Routing intake through procurement catches anything the agency buys. It does not catch a free browser tool a caseworker signs up for with a work email, so the inventory has to be paired with a route for people to disclose what they are already using without being punished for having started.

Six months in, the agency's AI inventory had grown from the four systems anyone could initially name to nineteen, every one with a named owner, a risk classification, and a documented approval. None of that existed when Gershom received his four-sentence memo. The structure was the deliverable. The policy could now stand on top of it, and the maturity self-assessment he had run on his first week had a defensible new answer.

Anti-Patterns

• Governance as bottleneck. Structures become so cumbersome that they prevent deployment: approvals take months, committees are too large and too slow. Avoid it by designing lean governance, pushing decisions to the lowest appropriate level, targeting approval in one to two weeks, and keeping committees focused.
• Governance without authority. Committees are created but cannot decide or enforce. They are advisory, and people feel free to ignore advice. Avoid it by making governance approval a precondition of deployment and attaching real consequences to non-compliance.
• Governance neglected by leadership. The structure exists but leadership treats it as compliance busywork. Avoid it by having leadership visibly support governance, participate in its meetings, and let it weigh in performance evaluations.
• Governance captured by one perspective. One constituency, technical or legal or programmatic, dominates and the others are marginalized. Avoid it by ensuring genuine representation and making room for disagreement rather than defaulting to "we will do what tech wants."
• Governance without evolution. The structure is designed, implemented, and never revisited while the organization and the technology move on. Avoid it by building in a review of governance effectiveness at least annually and adjusting.
• Treating a maturity level as a certification. A level is something an agency asserts about itself, not something a framework confers. Claiming Level 3 without producing the documented decisions that define Level 3 converts a diagnostic into a badge, and the badge then blocks the very conversation the diagnostic was for.
• Treating the intake form as shadow-AI prevention. Intake registers what someone chooses to register and procurement enforcement catches what the agency buys. Neither reaches a free tool adopted quietly by a team. Assume unregistered use exists, and build a low-friction disclosure route rather than concluding the form has closed the gap.
• Naming an owner who cannot act. A named Model Owner with no budget, no access to the vendor, and no authority to suspend the system satisfies the documentation and none of the accountability. Check that each named owner holds a lever, not just a line in a register.

Practice Prompts

1. Governance maturity assessment. Place your organization on the four levels described above. Where are you, where do you want to be, and what specifically is required to advance? For each level you claim, name the artifact that proves it.
2. Structure design. Design a governance structure for your organization. What size profile do you fit? Which committees are genuinely needed and which are ceremony? What roles must exist, and which of them will one person hold? Document it.
3. Decision rights. Build a decision rights framework using the grid above as a starting point. What decisions can be made at what levels? Under what conditions is escalation required rather than optional?
4. Implementation planning. Draft an implementation plan for establishing governance. What is the timeline, what resources are needed, and what are the phases? Compare your timeline against the size profile that matches you.
5. Stakeholder communication. Design a communication and change management approach for introducing new governance. How would you explain it to a program director, a data scientist, and a general counsel differently? How would you address resistance from each?

Reflection

Take three minutes with your own organization. What governance structures currently exist for AI, and are they formal or ad hoc? Who is actually responsible for making AI decisions, and is that clear to everyone or only to the people who wrote it down? When a problem is identified with an AI system, what happens next, and is there an escalation path a frontline employee could name without looking it up? If you were designing governance from scratch, what would you change about your current structure, and what would you keep?

Glossary

• Governance maturity. The level of formality and sophistication of an organization's governance processes, ranging from ad hoc to optimized. Self-assessed, not conferred.
• Chief AI Officer. Senior executive responsible for organizational AI governance strategy and compliance. Required of covered federal agencies by OMB M-24-10.
• Decision rights. The explicit allocation of who can make what decisions, and the conditions under which a decision must be escalated.
• Steering committee. Senior leadership committee that sets AI strategy and makes high-level governance decisions.
• Governance authority. The power of a governance structure to actually influence decisions and enforce standards, as distinct from its power to issue advice.
• Data Owner. The person accountable for a specific dataset's quality, provenance, access control, and retention.
• Model Owner. The person accountable for a specific system's performance, updates, monitoring, and compliance.
• Rights-impacting and safety-impacting. Risk categories drawn from OMB M-24-10 that determine the depth of review and the practices a system must satisfy before deployment.

Related Lessons

This lesson sits at the end of a governance sequence and depends on it. NIST AI RMF: The GOVERN Function and NIST AI RMF: MAP, MEASURE, MANAGE supply the functions your structure has to carry out. Risk Classification: Safety-Impacting vs. Rights-Impacting defines the tiers your decision rights table depends on, and Minimum Risk Management Practices defines what a high-risk approval must confirm. AI Use Case Inventory and Documentation (OMB M-24-10) is the register your intake feeds, Data Governance for AI covers the Data Owner role in depth, and Establishing an AI Governance Board goes further into standing up the board itself.

Closing

Governance is not separate from AI development; it is woven through it. Every stage of the lifecycle, from problem definition through deployment through ongoing monitoring, involves governance decisions and processes, and each of the pieces covered earlier in this chapter needs a structure to run inside. Risk classification needs someone to classify. Impact assessments need someone to read them. Minimum practices need someone empowered to insist. Data governance needs owners. None of it functions on its own.

What made Gershom's first six months work was not the elegance of his org chart. It was that four questions had answers a stranger could find: who decides, on what basis, who owns the result, and where is that written. Good governance makes AI projects more successful, and clear structures make project management possible, because the arguments that would otherwise be relitigated on every project have been settled once at the level where they belong.

Key Takeaways

• Structure precedes policy. A policy is a wish until someone is named to interpret it, empowered to enforce it, and accountable when it is ignored. Build the governance structure first.
• Assess maturity before designing. The ad hoc to optimized ladder tells you which structure is realistic. It is a self-description you must back with artifacts, not a rating anyone awards you.
• Match the structure to the size. Small agencies need one accountable person and a monthly committee. Large agencies need specialized committees, dedicated staff, and enterprise risk integration. Importing the large design into a small agency creates the bottleneck failure.
• The CAIO coordinates; the board decides. M-24-10 requires a Chief AI Officer, but consequential decisions belong to a cross-functional governance board, which protects both the decision and the decider.
• Every system and dataset needs a named owner with a lever. Governance fails when systems are owned by a contract or a team, and it fails just as quietly when the named owner has no authority to stop anything.
• Make decision rights explicit. Write down who approves deployment, data changes, scope expansion, and what happens when risk exceeds tolerance, tiered by risk classification rather than by sponsor enthusiasm.
• Governance must have real authority and real support. Advisory governance is ignored, and governance that leadership treats as busywork will be treated that way by everyone else.
• Design against decoration and against stasis. A standing cadence, a central inventory tied to the asset registry, an enforced intake route, durable documentation, and an annual review of governance itself keep the structure from becoming either theatre or a fossil.

Frequently Asked Questions

Our agency is small and nobody has AI in their title. Do we need all of this? No. You need the functions, not the org chart. In the 50-person example, an existing CTO took on the Chief AI Officer role, the data lead became Data Owner for everything, and technical staff owned their own systems. The whole design cost roughly 50 hours of CTO time plus 30 hours from other staff. What you cannot skip is naming someone, giving a body the authority to say no, and writing decisions down.

Where should the Chief AI Officer report? The lesson's large-agency example placed a newly hired CAIO under the CIO. That is one workable answer and not the only one. The test is whether the reporting line lets the role coordinate across privacy, civil rights, legal, procurement, and program without needing permission from the function whose systems it may have to stop. If the reporting line makes the CAIO subordinate to the office it must review, the structure has a defect regardless of the title.

How do we keep governance from becoming a bottleneck? Push decisions to the lowest appropriate level, keep committees small and focused, and set an explicit target for approval turnaround, for example one to two weeks. Tiered review does most of the work here: if every use case goes to the full board, the board becomes the queue. Reserve the heavy path for rights-impacting and safety-impacting systems and let low-risk tools take a lightweight check.

What if the governance board is advisory only? Then it is not governance yet. Advisory bodies are ignored precisely when it costs something to listen to them, which is the moment governance was for. The fix is structural rather than cultural: make governance approval a precondition of deployment, route procurement through intake so the path cannot be bypassed for anything purchased, and attach consequences to non-compliance.

How do we know our structure is working? Look for artifacts rather than activity. Can a stranger find out who approved a given system, on what conditions, and who owns it now? Does the inventory match what is actually running? Has the board ever changed an outcome, or has it approved everything put in front of it? A board that has never declined or conditioned a proposal is not proof that the proposals were good; it is a finding worth investigating.

How often should we revisit the structure itself? At least annually, as a scheduled review rather than a response to an incident. Assess whether the escalation paths were used, whether decision rights matched the decisions that actually arose, and whether the committee set still reflects where AI is being deployed. Governance that is designed once and never revised drifts out of contact with the organization it governs.