When a state's unemployment agency deployed an AI system to flag potentially fraudulent claims, the director, Eleanor Foss, expected applause for cutting fraud losses. What she got was a class-action lawsuit. The system had wrongly flagged tens of thousands of legitimate claimants, cut off their benefits during a recession, and the agency had built in no human review and no clear way to appeal. The Michigan MiDAS unemployment-fraud system is the documented version of this story: a determination accuracy problem severe enough that the state eventually had to refund affected claimants and overhaul the program. The technology was the smaller failure. The larger one was that the public learned the government had pointed an automated accuser at them with no recourse.

For an agency leader, public trust is not a soft outcome you hope follows good work. It is the operating license for everything you do, and it is asymmetric: it is built slowly, through years of ordinary competence, and it can be lost very quickly. That asymmetry is why a trust strategy has to run in two directions at once. The offensive half builds confidence deliberately while nothing is wrong. The defensive half prepares for the day something is, because something eventually will be. This lesson treats trust as measurable, buildable, and recoverable, and carries Eleanor's agency through all three.

Why trust is the real asset

A private company that loses customer trust loses market share. A government that loses public trust loses something harder to replace: the willingness of citizens to comply, to share data, to use the digital services you built, and to give you the benefit of the doubt when something goes wrong. Low-trust agencies pay for it everywhere, in more appeals, more manual fallback, more litigation, and slower adoption of the very tools meant to create efficiency. In the private sector, lost trust costs you customers. In government, it costs you the consent that makes governing possible.

There is a hard version of this claim worth stating plainly, because it sets the stakes for everything that follows. Without public trust, government AI is not merely unpopular; it becomes both illegitimate and ineffective. Illegitimate because a public authority exercising consequential power over people who do not accept its right to do so is operating on force rather than consent. Ineffective because the practical machinery of a low-trust system, the appeals, the workarounds, the refusals, the litigation, consumes more staff time than the automation ever saved. High-trust systems get used. Low-trust systems get fought.

What trust is not: the artifact trap

The most expensive mistake in this field is the belief that trust is produced by artifacts. Agencies publish a transparency report, stand up an algorithm register, run a public consultation, or post a model card, and treat the act of publishing as the discharge of an obligation. It is not. A published register tells the public what you did; it does not make what you did acceptable, and it does not transfer any part of the decision to the people who read it. The artifact is evidence that you looked. It is not absolution for what you found, or for what you did not.

Hold the same standard against every control in this lesson. A completed checklist has never once made a false statement true. A consultation informs a decision that your agency still owns, and the fact that people were asked does not make an unpopular outcome legitimate or a harmful one safe. An explanation makes a decision contestable, which is genuinely valuable, but it does not make the decision correct. Each of these things removes a specific reason for the public to distrust you. Removing reasons to distrust is necessary work and it is not the same act as earning trust, which happens over time and only through behavior.

The two halves of a trust strategy

Because trust is slow to build and fast to lose, the two halves of the strategy are not variations on one activity. The offensive half is everything you do while things are working: disclosing before you are asked, building recourse into systems that do not yet have complaints, publishing error rates nobody has demanded, and accumulating the ordinary record of reliability that people draw on when they decide whether to give you the benefit of the doubt. This work is unglamorous, never produces a visible win, and is almost always the first thing cut when budgets tighten.

The defensive half is the capability to respond when something breaks. It includes knowing in advance who announces a failure, how fast a system can be suspended, who can authorize reversing decisions in bulk, and what remediation you are prepared to offer before lawyers have negotiated it. Agencies that have to invent this under pressure invent it badly, because the instinct under pressure is to minimize and defend. Both halves are required. An agency that only builds is undefended, and an agency that only prepares for crises never accumulates the credit that lets it survive one.

Measuring trust before you assume it

Most leaders manage trust by anecdote and find out it is gone only when it makes the news. You can do considerably better by tracking a small dashboard of leading and lagging indicators, reviewed by someone senior enough to act on them. None of these measures is trust itself, which is a disposition inside people's heads that no dashboard observes directly. They are signals that correlate with it, and their value is that they move before the lawsuit does.

• Direct measures. Periodic public surveys asking specifically about confidence in the agency's AI-assisted decisions, not just general satisfaction with service. Satisfaction and trust diverge, and the gap is informative.
• Behavioral measures. Adoption rates of optional AI-powered services, opt-out rates where opting out is allowed, and the share of people completing tasks online rather than demanding a human. What people do is more reliable than what they tell a surveyor.
• Friction signals. Appeal rates, complaint volume, error-correction requests, and how long each takes to resolve. A rising appeal rate is the earliest trust warning most agencies have and the one most often filed as an operations metric nobody escalates.
• External scrutiny. Investigative press, advocacy-group attention, oversight inquiries, and legislative interest. These lag badly, but treating them as public relations problems rather than data is how a problem becomes a crisis.

Eleanor's agency had every one of these signals available and tracked none of them for the fraud system. The appeal rate had spiked months before the lawsuit was filed, sitting in a weekly operations report that went to a manager whose job was throughput. The data was screaming and nobody had built the path for it to reach a decision-maker. Building that path costs almost nothing compared to what it prevents, and it is the single highest-return item in this lesson for an agency that already has AI in production.

Building trust on purpose

Trust in government AI rests on a few foundations the public can actually feel, even if they never read your policy. Drawing on the principles behind federal guidance on agency AI use and the accountability frameworks published by government audit bodies, the practical version comes down to four commitments. What makes them work is that each one is checkable by an outsider, which is what distinguishes a commitment from a value statement.

• A human is accountable. No consequential decision about a person is final without a named human able to review and reverse it. For benefits, eligibility, and enforcement, automation can assist, but it cannot be the last word. The named part matters; an accountable office with no accountable person inside it is a diffusion of responsibility, not an assignment of it.
• People can understand and contest. Anyone affected gets a plain-language account of what happened and a usable path to appeal. An appeal process that takes ninety days and a lawyer is not a usable path, and a right to contest that only sophisticated claimants can exercise sorts your population by resourcefulness rather than by merit.
• The system is tested for fairness before and after launch. You check whether error rates differ across communities, and you keep checking, because models drift and populations change. A launch-time fairness result describes a system that no longer exists.
• You are honest about what the AI does. Tell people an automated system is involved and what it does, before anyone has to ask. Disclosure does not create trust on its own, but concealment that surfaces later destroys it, and the discovery is always worse than the disclosure would have been.

Had Eleanor's system launched with mandatory human review before any benefit was cut, and a real appeal path, the same false-positive rate would have produced a recoverable problem instead of a catastrophe. That is not the same as harmless. A wrongly suspended benefit is not an inconvenience to a household living on it, even when it is reversed quickly. The point is that recourse changes the character of an error from a permanent injury inflicted by an unreachable system into a mistake a person can get corrected. The error was technical. The betrayal was the absence of any way to fix it.

Transparency, procurement, and what disclosure actually buys

Disclosure obligations do not stop at the model. A significant share of trust failures in government AI begin not with an algorithm but with a contract: a vendor relationship, a data-sharing arrangement, or a system capability that was never disclosed and then surfaced through journalism or litigation. The damage in those cases comes less from what the contract permitted than from the discovery that the public had not been told. Procurement is where a great deal of consequential AI policy is actually set, and it is usually the least visible part of the process to the people the systems will be used on.

The practical implication is to treat contract-level transparency as part of the trust perimeter rather than as a procurement housekeeping matter. Publish what systems you have bought, what they do, which populations they touch, and which vendor operates them, on the schedule you would want if you were the person being assessed. That publication does not settle whether the arrangement was appropriate, and a leader who treats a published contract register as a defense has fallen back into the artifact trap. It removes the ambush. Removing the ambush is worth a great deal, because trust rarely survives the sequence in which the public learns something important from someone other than you.

Recovering from a trust failure

Sooner or later something will go wrong. How you respond determines whether it becomes a footnote or a defining scandal, and the instinct to minimize and defend almost always deepens the damage. The sequence below is ordered deliberately, because each step is harder to take credibly once you have skipped the one before it. An agency that fixes a system quietly and only acknowledges the failure after a reporter proves it gets no credit for the fix.

1. Acknowledge fast and specifically. Name what happened and who was harmed before the press does it for you. Vague regret reads as guilt management; specificity reads as honesty, and it is also the only version anyone can act on.
2. Stop the harm. Suspend the system or add human review to every affected decision while you investigate. Continuing to run a system you believe is broken while you study whether it is broken is indefensible and will be quoted back to you.
3. Make people whole. Restore benefits, refund, reverse decisions. Michigan ultimately had to pay back affected claimants, and doing that voluntarily and early is far cheaper than doing it under court order, in both money and standing.
4. Fix the system, visibly. Publish what you changed and what it now prevents. Publishing the fix does not restore trust by itself; it removes the reasonable fear that the same failure is still live, which is the obstacle standing in front of any recovery.
5. Invite outside eyes. An independent review or oversight audit signals you are not grading your own homework. Its value depends entirely on the reviewer's independence and on your publishing what they find, including the parts you dislike. A review commissioned, scoped, and shelved is worse than none, because it converts a failure into a documented one you can be shown to have ignored.

An international mirror

It helps to see how other systems have handled the same tension. The Netherlands lived through a near-identical story when an automated childcare-benefits fraud system wrongly accused thousands of families, with discriminatory effects, and the fallout contributed to a government resignation. Set beside Michigan, the comparison is instructive precisely because the technologies, the legal systems, and the political cultures differ so much and the failure shape is the same: automated accusation at scale, applied to people with little power, with no proportionate route to challenge it.

The positive cases are quieter and slower. New Zealand ran a public deliberation process on government use of algorithms, asking citizens directly which uses they considered acceptable. Deliberation of that kind improves the quality of the decision and gives officials a defensible account of why they drew a line where they did. It does not transfer ownership: the agency still owns the choice, and consulting the public on a system that later harms people is not a defense. Estonia's standing is likewise the product of many years of sustained investment in security, transparency, and reliability, which is an asset built at the pace of decades and damaged at the pace of a news cycle.

A third case sits between the two and is the one most likely to describe your own agency. A national government contracted for AI systems without adequate public disclosure of the arrangement, and when the relationship later became public the disclosure itself was the breach: not the technology, not a wrongful decision, but the fact that citizens learned about a consequential contract from somewhere other than their government. Nothing in that story requires a system to have performed badly. It is worth sitting with, because contract transparency is the cheapest item on this entire list and the one most often deferred on the reasonable-sounding grounds that the deal is commercially sensitive and can be described once it is settled.

The consistent lesson across borders is that the technology rarely causes a trust collapse on its own. Collapse comes from scale without recourse, from opacity, and from slow acknowledgment. Agencies that pair AI with human accountability, transparency, and fast correction give themselves a genuine chance of holding public confidence when individual decisions go wrong. They are not guaranteed it. A sufficiently severe harm, or one that lands on a community already convinced the state does not act in good faith, can overwhelm every good practice on this list, which is an argument for building the practices earlier, not for skipping them.

A public-trust scorecard for an AI system

Before launch and on a regular schedule after it, score each system against the questions below. Any "no" is a trust liability with an owner and a due date. Use this as a diagnostic that generates work, never as a certification: a clean scorecard describes the controls you have, not the outcomes your system produces, and every failure in this lesson happened at an agency that could have answered most of these questions correctly on paper.

• Accountability. Is a named human able to review and reverse every consequential decision? Yes or no.
• Recourse. Can an affected person appeal in plain language within a reasonable, published timeframe, without needing a lawyer? Yes or no.
• Fairness. Have we measured error rates across communities at launch and on a schedule since? Yes or no.
• Transparency. Do affected people know an automated system is involved and roughly what it does, and is the contract behind it disclosed? Yes or no.
• Monitoring. Are appeal rates, complaints, and opt-outs on a dashboard that a leader with authority actually reviews? Yes or no.
• Response readiness. Do we have a written plan to acknowledge, stop, and remediate a failure in days rather than months, with named owners? Yes or no.

Anti-Patterns to Avoid

• The register as discharge. Publishing an algorithm register, transparency report, or model card and treating publication as the obligation met. The register tells people what you did. It does not make what you did acceptable, and it moves no part of the responsibility onto the reader.
• The artifact as absolution. Pointing at a completed impact assessment, scorecard, or ethics review when a system is challenged. A completed checklist has never once made a false statement true, and an assessment that missed the harm is evidence about your process, not a defense of your outcome.
• Consultation as consent. Treating a public engagement exercise as authorization for whatever follows. Consultation informs a decision your agency still owns, and the people who attended did not agree to be harmed by the version you eventually shipped.
• Explanation as correctness. Assuming that because a decision can be explained, it must be right. Explanation makes a decision contestable, which is valuable and different. A clearly explained wrong decision is still a wrong decision, delivered more articulately.
• Trust metrics that measure satisfaction. Reporting service satisfaction scores as evidence of trust in AI-assisted decisions. People routinely report satisfaction with a fast service while distrusting the judgment behind it, and the two diverge most sharply in exactly the populations most affected.
• The appeal that only lawyers can use. Building a formally complete appeals process with timelines, forms, and evidentiary standards that no unrepresented person completes. On paper there is recourse. In practice you have sorted people by their access to help.
• Fixing quietly. Repairing a failure without acknowledging it, hoping the problem passes unnoticed. When it surfaces later, and it does, the concealment becomes the story and the fix earns nothing.
• The shelved independent review. Commissioning outside scrutiny, narrowing its scope, and declining to publish the findings. This converts a failure into a documented failure you can be shown to have known about, which is a substantially worse position than where you started.

Practice Prompts

• Find your unwatched signal. For one AI-assisted decision process in your organization, trace where appeal rates and complaint volumes are recorded and identify the most senior person who sees them on a routine basis. If that person cannot suspend the system, you have found a gap worth closing this month.
• Write the acknowledgment now. Draft the public statement you would issue if your highest-stakes AI system were found to have produced systematically wrong outcomes for a specific group. Write it while nothing is wrong, then note which facts you would need and cannot currently obtain quickly.
• Walk the appeal. Have someone unfamiliar with the system attempt to contest a decision using only the materials a member of the public receives. Record where they stall, how long it takes, and what outside help they needed.
• Score the artifact trap. List every transparency artifact your organization publishes about AI and, for each, write one sentence describing what a member of the public can actually do as a result. Artifacts with no answer are candidates for replacement by something actionable.
• Comparative post-mortem. Pick a documented government AI failure from another jurisdiction and map its sequence against your own systems. Identify which of its preconditions, scale, opacity, absent recourse, slow acknowledgment, exist in your organization today.

Reflection

Think about the AI-assisted system in your organization that touches the most vulnerable population. If it produced a serious, systematic error tomorrow, how long would it take you to find out, who would tell you, and how long after that would the first correction reach the first affected person? Consider what your organization currently offers as evidence of trustworthiness, and ask honestly whether each item changes what someone affected can do, or only what your organization can say. The distance between those two lists is the real state of your trust position.

Glossary

• Public trust. The willingness of citizens to accept an authority's decisions, share information with it, and extend the benefit of the doubt. It is a disposition held by people, not a property of a system.
• Lagging indicator. A signal that confirms an outcome after it has occurred, such as press coverage, litigation, or an oversight inquiry.
• Recourse. A usable route by which a person affected by a decision can have it reviewed and reversed, measured by whether unrepresented people actually complete it.
• Leading indicator. A signal that moves before the outcome it predicts, such as a rising appeal rate ahead of a formal complaint or lawsuit.
• Algorithm register. A published inventory of the automated systems an organization uses and what they do. A disclosure instrument, not an approval mechanism.
• Remediation. The act of restoring people to the position they would have been in absent the failure, including reversing decisions and repaying what was wrongly withheld.

Related Lessons

• Transparency: Citizens' Right to Know develops the disclosure obligations sketched here into a fuller account of what the public is entitled to about government AI.
• Building Trust Through Transparency examines the specific mechanisms by which openness supports confidence, and where it stops working.
• Public Reporting and Algorithmic Transparency covers the practical design of registers, reports, and disclosures that people can act on.
• Crisis Management for AI Failures takes up the defensive half of the strategy in detail, from detection through public response.
• When Government AI Goes Wrong works through documented failure cases and the conditions that produced them.
• Accountability Frameworks for AI Failures addresses who answers for what when an automated system harms someone.
• Community Engagement in Government AI covers consultation and deliberation as practices, including how to run them without treating them as consent.
• AI and the Social Contract steps back to the underlying question of what citizens are owed when a state automates its judgment.

Closing

Eleanor's agency eventually rebuilt the fraud system, and the version that returned looked ordinary: a human decision before any benefit stopped, a plain-language notice, an appeal that a person could finish without help, error rates measured by community and published, and an appeal-rate dashboard on the director's own weekly review. None of that is innovative, and none of it would have made a headline. It is simply the difference between an agency that can make a mistake and an agency that can only inflict one. Trust is not a message you send about your systems. It is the accumulated record of what happens to people when your systems are wrong.

Key Takeaways

• Trust is the operating license. Lost trust raises costs everywhere through appeals, litigation, and refusal to use services, and without it government AI is both illegitimate and ineffective.
• Artifacts are not trust. A register, report, consultation, or checklist removes a reason to distrust you; none of them discharges an accountability obligation or makes a decision correct.
• Run both halves of the strategy. Build confidence deliberately while things work, and prepare the response capability before you need it, because trust is slow to build and fast to lose.
• Measure trust before you lose it. Surveys, adoption, opt-outs, and above all appeal rates give early warning, but only if they reach someone with the authority to stop a system.
• Recourse is what changes the character of an error. A named human able to reverse a decision and an appeal an unrepresented person can finish turn permanent injuries into correctable mistakes.
• Disclose before you are asked, including contracts. Concealment that surfaces later does more damage than the disclosure would have, and procurement is where much of your AI policy is actually set.
• Recover by acknowledging fast, stopping the harm, and making people whole. Each step is harder to take credibly once you have skipped the one before it, and fixing quietly earns nothing.
• Outside scrutiny helps only if it is independent and published. A review that is narrowly scoped or shelved leaves you worse off than none at all.

Frequently Asked Questions

Can we measure public trust directly? Not really. Trust is a disposition inside people, and every practical measure is a proxy: surveys capture what people are willing to say, behavioral measures capture what they do under the options you gave them, and friction signals capture what happens when they are unhappy. Use several at once and watch the divergences, because the gap between reported satisfaction and observed opt-out behavior is usually where the real story is. Treat any single number claiming to be a trust score with suspicion.

Does publishing more information always help? No. Publication helps when it gives someone a specific thing they can act on and when it arrives before anyone had to demand it. Volume without actionability produces a documented but still unaccountable system, and it can make things worse by demonstrating that you knew. The test to apply to each disclosure is simple: what can an affected person now do that they could not do before? If the honest answer is nothing, the disclosure is decoration.

Our system is accurate. Do we still need appeals? Yes, and accuracy is the wrong frame. Aggregate accuracy says nothing about the individual in front of you, error rates differ across groups even in accurate systems, and the population your model sees will drift away from the one it was tested on. Beyond that, recourse is not only an error-correction mechanism. It is the thing that tells people the state is still answerable to them, which is a large part of what they are actually assessing when they decide whether to trust you.

What if consultation produces an answer we cannot act on? That is a normal outcome and it should be handled by saying so. Consultation informs a decision your agency continues to own, and pretending otherwise damages both the decision and the practice. Report back on what you heard, state which parts you adopted and which you did not, and give reasons for the second category. Communities are considerably more tolerant of being overruled with an explanation than of being consulted and then ignored in silence.

How do we justify trust investment when nothing is going wrong? By pricing the alternative. The costs of low trust are real and already showing up in your budget as appeal volumes, manual fallback, litigation exposure, and stalled adoption of systems you have already paid for. The offensive half of a trust strategy never produces a visible win, which is why it is cut first and why the agencies that keep funding it are the ones that have already survived a failure. Borrowing that lesson secondhand is cheaper than earning it.