When Lena Okafor became chief data officer at a large state revenue department, she inherited a problem nobody had named: the agency ran nine AI systems, and bias was being checked on exactly zero of them. Not because anyone opposed fairness, but because checking was somebody's side project, done once, by hand, on one system, eighteen months ago. Lena's predecessor had bought a fairness toolkit, run it once on the fraud model, gotten a clean-ish result, and moved on. Meanwhile the model kept learning from new data, the population kept shifting, and nobody was watching. Lena's job was not to check one model once. It was to build a machine for checking every model, forever.

That shift, from a heroic one-time audit to a continuous organizational process, is what "at scale" means. Anyone can run a bias test on a single system in a workshop. The hard, leadership-level problem is making bias detection a routine that runs across dozens of systems, catches drift over time, and triggers fixes without depending on one diligent person remembering. This lesson is about that machine: the organizational structure, the detection process, the tooling, the remediation workflow, and the monitoring that keeps it alive.

Why One-Time Checks Fail at Scale

A single bias audit is a photograph. The systems it studies are movies. Three forces make yesterday's clean result meaningless tomorrow:

• Data drift. The population the model sees changes. A tool calibrated on last year's applicants slowly miscalibrates as the applicant mix shifts.
• Model updates. Vendors retrain and update models, sometimes silently. The system you audited is not the system running today.
• Scale of inattention. With nine systems and one part-time checker, eight systems are always unwatched. Bias does not wait for your audit calendar.

A single AI system might have bias. A portfolio of AI systems spread across an organization needs something different in kind: organizational structures, tools, metrics and governance that operate whether or not anyone is paying attention this quarter. At scale, bias detection and mitigation stop being assessments and become operational processes, which is a change in who owns the work as much as in how often it runs.

Government agencies feel this harder than most, because a single agency often deploys AI across several domains serving very different populations at once. Civil rights requirements mandate proactive identification and elimination of discrimination, so the obligation does not wait for a complaint to arrive. The frameworks Lena leaned on say the same thing in different words. The NIST AI Risk Management Framework, a voluntary standard, treats measurement and management as ongoing functions rather than one-time gates. The federal guidance on rights-impacting AI expects continuous monitoring after deployment, not a single pre-launch blessing. Both assume the work is a loop, not a line.

What Bias Detection Can and Cannot Tell You

Before building the machine, be precise about what it produces, because the most dangerous moment in this work is a clean test result. Systematic detection makes discrimination far more likely to be found, and finding it early is the whole point. It does not prevent discrimination, and a portfolio that has been tested is not thereby a portfolio that is fair.

Disaggregation surfaces only the groups you thought to compare, on the metrics you thought to compute, in the data you happened to have. If your records do not capture disability status, no amount of testing will reveal a disability-related disparity. If you compared race and gender but not language or geography, a language-based harm stays invisible while every chart on the dashboard stays green. If your chosen metric is selection rate and the harm lands in error rates, the metric will not move. A clean result is evidence about what you measured. It is not a finding about the system as a whole, and it is certainly not a finding about the people your data does not describe.

This is why qualitative input from affected communities belongs alongside the numbers rather than after them. Metrics can look good while the lived experience of the people the system acts on is bad, and the gap between those two is exactly where a measurement programme fails silently. Treat a clean test as a reason to ask which comparison you have not run yet, not as a conclusion.

Who Owns Fairness: Structure and Governance

Fairness at scale requires organizational structure, not just good intentions. The roles below distribute the work so that no single person's diligence is the control. Executive accountability for fairness across the organization sits with a named senior owner, sometimes titled a chief AI fairness officer. A fairness engineering team holds the technical expertise to design and implement bias detection. The civil rights office provides oversight and compliance judgment, which is a different function from the engineering one and should not be collapsed into it. Program teams remain responsible for the fairness of their own specific systems, because ownership that sits only in a central team becomes advisory.

Governance gives that structure a rhythm. A fairness committee meets monthly with representatives from civil rights, from each major AI system, and from fairness engineering. Executive reporting on fairness metrics across the portfolio runs quarterly, so leadership sees every system side by side rather than one at a time in a crisis. An independent third party audits fairness annually. The fairness policy itself is reviewed and updated annually as needed, because the policy ages as fast as the systems do.

None of that runs on goodwill. Budget the work explicitly: fairness engineering resources covering staff, tools and training; regular audits and evaluations; and remediation efforts for identified bias. That last line is the one agencies most often omit, and omitting it produces the worst possible posture, which is an organization that reliably detects problems it has no funded way to fix.

Building the Detection Process

Lena's first move was to make bias detection a defined, owned, repeatable process rather than an act of individual heroism. A scalable process has four standing parts:

• A registry of what to test. Every rights-impacting and safety-impacting system, drawn from the agency's AI inventory, with a named owner and a defined test cadence.
• A standard test battery. The same core fairness metrics applied to every system, so results are comparable and nobody reinvents the method each time.
• Defined groups and thresholds. Agreed-upon population groups to test across, and agreed-upon thresholds that separate "acceptable" from "investigate" from "stop."
• A trigger calendar. Tests run on a schedule and on events: every model update, every major data change, plus a regular interval even when nothing obvious changed.

The monitoring rhythm underneath is straightforward. Run bias detection on each active system on a monthly fairness audit. Compare every result to the system's own baseline, asking whether fairness has improved, degraded or held steady, because an absolute number without a trend tells you very little. Report disaggregated, showing performance for each group rather than a portfolio average. And alert on drift: when a fairness metric degrades beyond the recorded threshold, escalate automatically rather than waiting for the next scheduled review.

The Metrics, in Plain Language

You do not need a statistics degree to direct this work, but you should know what your team is measuring, and you should be able to state each check operationally rather than by label. Labels travel badly between teams; the underlying comparison does not.

• Selection-rate comparison, also called demographic parity. Does the system approve, flag, or select one group at a meaningfully different rate than another? Stated as a target, it asks for equal acceptance or approval rates across groups.
• Error-rate comparison, also called equalized odds. Are the system's mistakes concentrated in one group? The check compares false positive and false negative rates across groups. A fraud flag that is wrong far more often for one community is doing real harm even when overall accuracy looks fine.
• Calibration. When the system says "80% likely," is that actually true for every group, or only for the group it was mostly trained on? The check asks whether a stated confidence level is borne out at that level for each group separately.
• Representation. Are all relevant groups present in the training data at all? This is a data-coverage check rather than an outcome metric, and it belongs early: a group absent from the training data will often also be absent from your test results, which is how a gap becomes invisible rather than acceptable.

No single metric is "the right one," and they can conflict with each other in ways that have no purely technical resolution. Equalizing selection rates and equalizing error rates are not generally satisfiable at the same time. That means the choice of which to prioritize is a policy decision leadership must make deliberately, for each system, in light of what that system does and who bears its errors. Define fairness explicitly per system rather than assuming one definition fits the portfolio, and record the definition alongside the results so a later reader knows what was being tested.

Thresholds: What the Agency Commits To

Every part of this machine runs on thresholds, so it is worth being exact about what a threshold is. The variance you treat as acceptable across groups, the point at which a result moves from "acceptable" to "investigate," and the point at which it moves to "stop" are commitments your agency makes to itself. Set them in advance, in writing, before the results arrive, so they cannot be quietly relaxed to accommodate a number nobody wants to act on. Record who approved them and when.

A threshold set this way is an operating rule, not a legal test. Staying under it does not establish that a system is lawful, and crossing it does not establish that a system is unlawful. Those are questions for counsel on the specific facts of the specific programme, and a number chosen internally neither creates nor discharges them. What the recorded threshold does give you is a decision made calmly in advance rather than under pressure, and a trigger that fires without requiring anyone to volunteer bad news.

Selecting Tools

Lena's team standardized on shared fairness toolkits rather than nine bespoke scripts. Open-source options in common use include Fairlearn, IBM's AI Fairness 360, which also carries bias mitigation algorithms alongside detection, Google's Fairness Indicators, and the What-If Tool for exploring model behavior across demographic groups, usually paired with custom monitoring dashboards that track fairness metrics over time. The point of standardizing is not the specific library. It is that every system gets measured the same way, so a disparity in the fraud model and a disparity in the hiring screen are spoken in the same language and can be triaged by the same governance board.

When choosing tooling at scale, weigh four things. Does it cover the metrics your policy prioritizes, rather than the metrics it happens to compute by default? Can it run automatically on a schedule rather than only by hand, since anything requiring a person to remember will eventually not run? Does it produce output a non-technical governance board can read, because a result nobody on the committee can interpret will not drive a decision? And does it fit systems you buy as well as systems you build, since a vendor black box still needs its outputs tested even when you cannot see inside it.

The Detection and Remediation Workflow

Detection without a response is just documentation of harm. The most important thing Lena built was not the test; it was the workflow that fires when a test fails. A disparity should trigger a defined path, not a debate about who is responsible.

1. Discovery. Routine monitoring detects the issue, typically as a system that performs well overall, say 92% accuracy, but poorly for a specific group, say 78% accuracy for that group. The alert triggers when a fairness metric exceeds the recorded acceptable variance.
2. Triage. Confirm the disparity is real and material rather than a small-sample artifact, and classify its severity. A gap measured on a handful of cases is a prompt to gather more data, not a finding.
3. Contain. For severe rights-impacting disparities, add a human review step or pause automated action for affected cases while you investigate. Stopping harm comes before explaining it.
4. Investigate. The fairness engineering team finds the root cause. Is this algorithmic bias, where the system systematically disadvantages a group, or data bias, where the training data was already skewed? Other common sources are a proxy variable standing in for a protected characteristic, a decision threshold that fits one group poorly, and drift since deployment. Document the findings in a fairness incident report.
5. Determination. Decide whether this is a fairness violation requiring remediation, or an acceptable performance variance arising from legitimate operational factors. This decision is made jointly by the civil rights office and the program team, never by the engineering team alone, and the reasoning is written down. Treat "legitimate operational factors" as a conclusion you have to evidence, not a category you can assert.
6. Remediate. Apply the fix that matches the cause, and where a vendor owns the system, require them to correct it under the contract.
7. Implement and verify. Execute the mitigation, re-evaluate the fairness metrics to confirm the fix actually worked, and monitor for unexpected consequences of the mitigation itself, because a change made to close one gap can open another.
8. Document. Log the incident in a fairness register, record the root cause, record the remediation steps and their results, and capture the lessons that would prevent a similar issue elsewhere in the portfolio.

Remediation Strategies

The fix has to match the cause, which is why diagnosis precedes remediation in the workflow. Four families of mitigation are available, and most real remediations combine them.

Data-level mitigations work on the inputs: collect more training data from underrepresented groups, balance the training data toward equal representation across groups, adjust training data weights to emphasize underrepresented groups, and correct biased labels in the training data. Label correction is the least glamorous and often the most effective, because a model trained on decisions that were themselves biased will reproduce them faithfully.

Algorithm-level mitigations work on the model: add fairness constraints to model training, use fairness-aware algorithms that explicitly optimize for a fairness objective alongside accuracy, or use ensemble approaches combining multiple models in a fairness-aware way. Threshold adjustment, meaning the use of different decision thresholds for different groups, appears on this list in the source material and is genuinely used in practice, but treat it as a legal question before a technical one. Setting decision thresholds by protected group carries its own exposure, and it is not a knob your data team should turn without counsel signing off on the specific programme and the specific characteristic.

Operational mitigations work around the model, and they are often what you reach for first because they can be deployed immediately: human review for decisions affecting disadvantaged groups, escalation of borderline and high-uncertainty cases to manual decision-making, targeted fairness audits for specific populations, and transparency, meaning explaining to the affected person why the decision was made. Human review only mitigates if it is real review. A reviewer processing a queue of machine recommendations under time pressure, agreeing with nearly all of them, has become a rubber stamp, and the override rate is the number that tells you whether that has happened.

Portfolio mitigations work across systems, which is the level most agencies never reach. Ensure consistency, so related systems reach similar decisions on similar facts. Conduct holistic review, checking that the combination of systems does not disadvantage a group even when each one passes individually. And maintain an appeal process that lets a person dispute a decision, which is both a remedy for the individual and one of the few detection channels that catches harms your metrics were never designed to see.

Scaling Fairness Across the Organization

Four levers turn a working process in one office into an organizational capability. The policy framework comes first: fairness requirements that apply to all AI systems rather than being optional; fairness definitions and acceptable variance set per system because they vary by mission; a fairness assessment required before deployment; and ongoing monitoring required after it. Written this way, a program team cannot opt out by declining to have an opinion.

Training and capability building comes next. Everyone working on AI systems is trained on fairness concepts, so program managers can read a disaggregated report without a translator. The fairness engineering team is available for consultation rather than functioning as a gate. Tools and templates are standardized, and practices are documented and shared so the tenth system benefits from what was learned on the first.

Tooling and automation makes the process survive inattention: fairness monitoring runs automatically on all systems on a monthly schedule, alerts fire on fairness drift, dashboards exist separately for executives, program teams and the fairness team because those three audiences need different views, and the whole thing integrates with your existing incident management system rather than living in a parallel spreadsheet.

Accountability closes the loop. Program managers are accountable for the fairness of their systems, fairness metrics appear in performance evaluations, executive sponsorship is visible, and an annual fairness report goes to leadership. Accountability without the earlier three levers is unfair to the managers it lands on; the earlier three without accountability decay within a year.

Continuous Monitoring: Keeping It Alive

The final piece is what turns a process into a standing capability. Continuous monitoring means the tests run on their own, the results land on a dashboard the governance board reviews on a fixed cadence, and a threshold breach raises an alert without waiting for someone to remember to look. Lena set three monitoring habits: automated scheduled runs on every registered system, a quarterly fairness review where the board sees every system's status side by side, and an alert rule that escalates any breach of the "stop" threshold immediately. The goal is that no system is ever eighteen months unwatched again.

Monitoring detects what you chose to watch, on the cadence you chose to watch it. A harm affecting a group you are not disaggregating, or expressing itself in a metric you are not computing, will not appear on the dashboard however green the dashboard looks. Build in the channels that catch what the metrics miss: complaint tracking, appeal outcomes, and periodic qualitative engagement with the communities the systems act on. Those channels are noisier than a fairness metric and they are the only part of the system that can tell you about a harm you did not anticipate.

Lena's Result

A year in, Lena's nine systems were all on the registry, all tested on a schedule with the same metrics, all visible on one quarterly dashboard. The continuous monitoring earned its keep when the fraud model, the very one that had passed its single audit eighteen months earlier, drifted: its false-positive rate for one region crept past the investigate threshold as the regional economy changed. The alert fired, the workflow ran, the threshold was recalibrated, and the drift was corrected in weeks instead of being discovered years later in a complaint. The difference was not a smarter test. It was a process that never stopped looking.

Lena would be the first to say what the year did not prove. Nine systems now have a documented, comparable fairness posture on the groups the agency's data can describe, which is a great deal more than she inherited and less than a guarantee. The registry does not know about the harms nobody has thought to measure. That is why the appeal channel and the community engagement stayed on her list, and why the annual third-party audit exists to ask the questions the internal team has stopped asking.

Anti-Patterns

• Fairness as a launch checkpoint. The system is assessed before deployment and then assumed to stay fair, because fairness feels like a gate rather than an operation. Fairness degrades after deployment and the problem is found far too late. Make monitoring ongoing and treat fairness as an operational concern with an owner and a cadence.
• Nobody owns fairness. Responsibility is diffuse because adding fairness structures looks like overhead, and issues slip through until an external auditor finds them and the damage is reputational as well as substantive. Stand up a dedicated fairness function, name clear accountability, and give it a governance structure.
• Detecting without fixing. Bias is identified and not remediated, because fixing is hard and ignoring is easy. The complaint becomes a lawsuit and the organization is exposed with its own documentation showing it knew. Commit to a documented remediation process, and fund it, before you start detecting.
• Treating a clean test as proof of fairness. Disaggregation surfaces only the groups you thought to compare, on the metrics you thought to compute, in the data you happened to have. A green dashboard is evidence about those choices, not a finding about the system. Respond to a clean result by asking which comparison you have not run, not by closing the file.
• Aggregate metrics standing in for lived experience. Portfolio-level fairness numbers look healthy while the actual experience of people the systems act on is poor, because metrics are easier to produce than understanding. Report disaggregated by group, and pair the numbers with qualitative input from affected communities and with what the appeals and complaints are actually saying.
• Inventing your own legal threshold. An internally chosen variance is an operating rule your agency commits to in advance. Presenting it as the line between lawful and unlawful, in either direction, is a claim your agency is not in a position to make. Record it as a trigger, route the legal question to counsel on the specific facts, and never let a passed internal threshold be offered as a defence.
• Turning group-specific thresholds into a technical knob. Different decision thresholds for different groups is a real mitigation technique with real legal exposure attached, and it is the one item on the remediation list that should never be implemented on engineering judgment alone. Get counsel's position on the specific programme and characteristic first, and record it.
• Letting human review become a rubber stamp. Adding a human step is the fastest mitigation available and the easiest to hollow out. A reviewer clearing a queue under time pressure, agreeing with nearly every recommendation, has added latency rather than protection. Track the override rate, and treat a rate near zero as a finding about the review, not a reassurance about the model.

Practice Prompts

• Design the structure. Sketch the organizational structure for fairness at scale in your agency: who owns fairness, what the governance structure is in terms of meetings, decision rights and accountability, which fairness metrics you monitor and how often, and what escalation process fires when bias is detected.
• Design portfolio detection. Design a system for detecting bias across your organization's whole AI portfolio. Which metrics will you monitor, selection rate, error rate, calibration or others? What tools and automation will you use? What monitoring frequency, monthly, quarterly or real-time? What triggers escalation? How will results be communicated, and to whom?
• Design the response workflow. Work through detection and investigation, determination of whether this is bias requiring remediation, evaluation of data-level, algorithm-level and operational remediation options, implementation and monitoring, and finally documentation and learning. Name who decides at each step.
• Triage three scenarios. For each of the following, give a root cause hypothesis, remediation options, and the outcome you would expect: gender disparity in hiring recommendations, with women selected at 75% and men at 92%; racial disparity in approval rates, with white applicants at 85% and Black applicants at 68%; and age disparity in recommendations, with younger applicants receiving much more favorable treatment. For each, state which of your remediation options needs counsel involved before it is implemented.
• Draft the fairness policy. Write the general fairness requirements that apply to all AI systems, the fairness definitions in use, the acceptable variance and the reasoning behind it, the monitoring requirements covering frequency, metrics and tools, the remediation requirements covering process, accountability and timeline, and the governance rules for who decides and how disputes are resolved.

Reflection

Think about your organization's AI portfolio as it stands today. Who is accountable for fairness, by name, and would they say so if asked? How do you monitor fairness across systems rather than one system at a time? Which fairness metrics matter most for your highest-stakes system, and who chose them? If bias were discovered next week, what exactly would happen, and is that path written down anywhere or does it depend on who is in the room?

Then the harder pair. What organizational barriers actually prevent fairness work at your agency, and how would you address them? And which groups affected by your systems are invisible in your data, so that no test you run will ever tell you how they are being treated? The second question rarely has a comfortable answer, and the agencies that ask it are the ones that eventually find the harm before a complainant does.

Glossary

• Demographic parity. Equal acceptance or approval rates across demographic groups. Operationally, the check compares selection rates group by group.
• Equalized odds. Equal error rates across demographic groups, comparing false positive and false negative rates. Operationally, the check asks whether the system's mistakes fall disproportionately on one group.
• Calibration. Whether a stated confidence level is borne out at that level for each group separately, so that "80% likely" means the same thing for everyone.
• Representation. Whether all relevant demographic groups appear in the training data. A data-coverage property rather than an outcome metric.
• Fairness drift. Fairness metrics degrading over time as the data or the system changes.
• Algorithmic bias. A system systematically producing different outcomes for different demographic groups.
• Proxy variable. An input that stands in for a protected characteristic without naming it, allowing a disparity to persist in a model that never sees the characteristic directly.
• Fairness incident report. The documented record of a detected disparity, its investigation, root cause, remediation and verified result, held in a register across the portfolio.

Related Lessons

• Understanding AI Bias covers where bias originates, which is the ground this lesson's detection process stands on.
• Bias Detection Tools and Methods goes deeper into the individual techniques than a portfolio-level lesson can.
• Enterprise AI Risk Management places fairness monitoring inside the wider risk framework it has to report into.
• AI Red-Teaming Fundamentals covers adversarial testing, which finds a different class of failure than scheduled fairness metrics do.
• Privacy Engineering for AI matters here because the demographic data that makes disaggregation possible is itself sensitive and needs handling.
• Algorithmic Impact Assessments produces the pre-deployment baseline your continuous monitoring measures drift against.

Closing

Fairness at scale requires organizational commitment, dedicated resources, clear metrics and documented processes. That is what turns fairness from an aspiration into an operational reality: roles, governance, monitoring and remediation, with teams held accountable for the systems they own and metrics used to drive improvement rather than to reassure.

Build the capability so it detects continuously, responds through a defined workflow, and keeps looking at the systems nobody is currently worried about. Hold the results loosely enough to stay curious about what you have not measured, and keep the appeal channel and the community engagement open alongside the dashboard. This is how agencies protect the people their systems act on, meet their civil rights obligations, and keep the public trust that makes the next deployment possible.

Key Takeaways

• At scale means a process, not a heroic audit. The leadership challenge is making bias detection a routine that covers every system continuously, not a one-time check that depends on one diligent person.
• One-time checks expire. Data drift, silent model updates, and the sheer number of unwatched systems make yesterday's clean result meaningless today.
• Detection finds what you look for. Disaggregation surfaces only the groups you thought to compare, on the metrics you thought to compute, in the data you happened to have. Systematic testing makes discrimination far more findable; it does not make a tested portfolio a fair one.
• Fairness needs structure, not intentions. A named executive owner, a fairness engineering team, an independent civil rights office, and program teams that own their own systems, with a monthly committee, quarterly executive reporting and an annual third-party audit.
• Budget the remediation, not just the detection. An organization that reliably finds problems it has no funded way to fix is in a worse position than one that never looked.
• Standardize the method. A registry, a common metric battery, agreed groups and thresholds, and a trigger calendar let you compare and triage disparities across very different systems.
• Choosing which fairness metric matters is policy. Selection-rate, error-rate and calibration checks can conflict and cannot generally all be satisfied at once; leadership must deliberately decide which harms to prioritize, per system, and record the choice.
• Thresholds are commitments, not legal tests. Set the acceptable variance in advance and in writing so it cannot be relaxed to fit an unwelcome result. Clearing it does not establish lawfulness; crossing it does not establish unlawfulness. Those questions go to counsel.
• Standardize tooling for comparability. Shared toolkits across systems let one board read every result in the same language, including for vendor black boxes whose internals you cannot inspect.
• Build the remediation workflow first. Detection without a defined discovery-triage-contain-investigate-determine-remediate-verify-document path is just documentation of harm, and containing severe disparities comes before explaining them.
• Match the fix to the cause, and route the legal ones. Data-level, algorithm-level, operational and portfolio mitigations each address different root causes. Group-specific decision thresholds need counsel before implementation, and human review only mitigates if the override rate shows it is real.
• Make monitoring automatic, and keep the human channels open. Scheduled runs, a quarterly board dashboard and threshold alerts stop systems going unwatched; complaints, appeals and community engagement are what catch the harms your metrics were never designed to see.

Frequently Asked Questions

Our fairness tests come back clean. Are we done?

No, and this is the result to be most careful with. A clean test says the groups you compared, on the metrics you computed, in the data you had, did not show a disparity above the variance you recorded. It says nothing about a group your data does not identify, a metric you did not run, or a harm that expresses itself somewhere other than in selection rates. Treat a clean result as a prompt to ask which comparison is missing, and keep the complaint and appeal channels open as an independent signal.

Which fairness metric should we standardize on?

There is no single correct answer, and the metrics genuinely conflict, so this is a decision leadership makes rather than one your data team can settle. Ask what the system does and who absorbs its errors. A tool that flags people for investigation puts the weight on error rates, because a false positive is an innocent person under scrutiny. A tool that allocates a scarce benefit puts more weight on selection rates. Decide per system, write down the definition you chose and why, and store it with the results so a later reader knows what was tested.

We do not collect demographic data. How can we test at all?

Start by recording that limitation explicitly, because it is a finding about your monitoring programme and not a neutral fact. Then work with what exists: geographic and program-level breakdowns, appeal and complaint patterns, and testing on your own historical case records. Involve your privacy office and counsel early, because collecting sensitive attributes for fairness testing raises real questions about purpose, retention and access. What you should not do is let the absence of data read, later, as though fairness had been checked.

Can we just adjust the thresholds for the disadvantaged group?

Not on engineering judgment. Different decision thresholds for different groups is a genuine mitigation technique and it carries legal exposure that varies by programme and by characteristic. Take it to counsel with the specific facts before anyone implements it, and record their position. In the meantime, operational mitigations such as targeted human review and escalation of borderline cases can usually be deployed immediately while the underlying cause is diagnosed.

The disparity is small and might just be noise. What do we do?

That is exactly what the triage step is for. Confirm whether the result is material or a small-sample artifact before you escalate, and if the sample is thin, the action is to gather more data rather than to close the item. Do not let "it might be noise" become the standing response to every uncomfortable number: record what sample size would settle it, and set a date to look again.

Who should make the call on whether a disparity is a violation?

The civil rights office together with the program team, not the fairness engineering team alone. The engineering team establishes what the numbers are; whether a variance reflects a legitimate operational factor or a violation is a judgment that needs compliance expertise and programme knowledge, and it should be written down with its reasoning. Treat "legitimate operational factor" as a conclusion you have to evidence, because it is the phrase under which unexamined disparities most often get filed away.