AI bias audit: guide, methodology, and free checklist

An AI bias audit is a structured, independent evaluation of an artificial intelligence system to determine whether its outputs produce disparate, and potentially discriminatory, outcomes across protected demographic groups. Those groups typically include race, sex, age, disability status, national origin, and in some jurisdictions sexual orientation or religion.

A bias audit is conducted by a qualified third party with no commercial stake in the system's continued deployment. The auditor examines the system at three levels: the training data used to build the model, the statistical outputs of the model under controlled testing conditions, and, where accessible, real-world decision records that reveal disparities emerging after deployment.

The core analytical tool is disparate impact analysis, borrowed from employment discrimination law. Under the four-fifths rule in EEOC guidance, a selection rate for any protected group that falls below 80% of the highest group's rate signals potential adverse impact requiring explanation or remedy. Modern audits extend past that single metric to false positive rate parity, calibration across groups, and counterfactual fairness: testing whether changing only a protected attribute, while holding everything else constant, changes the outcome.

AI systems can encode and amplify discrimination without discriminatory intent. A resume screening model trained on historical hiring data inherits the biases in those past decisions. A credit scoring algorithm that uses zip code as a feature can replicate the effects of redlining. Quantifying those disparities, and documenting what was done about them, is a legal obligation in several jurisdictions and a defensible risk management practice everywhere else.

A rigorous audit runs in five phases. Each builds on the previous one, and the full audit produces a written report with quantified findings, identified disparities, and prioritized remediation recommendations.

Phase 1: Data audit

The audit begins with the training data, because a model is only as fair as the data it learned from. Auditors examine the composition of training datasets for representation gaps, historical bias encoded in outcome labels, proxy variables that substitute for protected characteristics (zip code for race, name for national origin), and data quality disparities across groups.

A thorough data audit traces lineage: where the data came from, what transformations were applied, and whether preprocessing introduced or amplified bias. Auditors look for missing data patterns that correlate with protected group membership, since sparse data on minority groups is itself a risk factor for biased outputs.

Phase 2: Model testing and disparate impact analysis

Statistical testing is the core of the audit. Auditors run the system on a test population with known demographic attributes and measure outcomes across protected groups. Standard metrics:

• Selection rate by protected group (four-fifths rule threshold: below 0.8 signals adverse impact)
• Scoring rate and scoring-rate impact ratio, where the tool produces scores rather than selections
• False positive rate parity: are false positives distributed equally across groups?
• False negative rate parity: who is incorrectly rejected at higher rates?
• Calibration: do confidence scores mean the same thing across groups?
• Counterfactual fairness: does changing only a protected attribute change the outcome?

NYC LL 144 requires selection or scoring rates and impact ratios for sex categories, race and ethnicity categories, and intersectional categories, with categories under 2% of the data excludable if disclosed. EU AI Act conformity assessments require technical robustness testing for the relevant high-risk categories.

Phase 3: Documentation review

Auditors review model cards, data sheets, intended use specifications, known limitations disclosures, and consumer-facing notices, assessing whether the system's actual behavior matches its stated purpose. Gaps are common when models drift after deployment or when vendor documentation misses real usage contexts.

For third-party systems, the auditor examines vendor contracts to determine whether the deployer received the information it needs to meet its own obligations.

Phase 4: Remediation recommendations

The report identifies disparities and ranks remediation options by feasibility and expected impact. Common pathways:

• Data resampling or reweighting to correct demographic imbalances in training data
• Threshold adjustment: different decision cutoffs per group to equalize outcomes
• Feature removal or transformation: eliminating or transforming proxy variables
• Model retraining with fairness-aware objectives (in-processing interventions)
• Post-processing calibration: adjusting model outputs after scoring

No single fairness criterion satisfies every objective at once; that is a mathematical result known as the impossibility theorem of fairness. A qualified auditor makes the trade-offs explicit and helps document the policy justification for the approach chosen.

Phase 5: Ongoing monitoring

A point-in-time audit is necessary but not sufficient. Systems develop new disparities after deployment as the input distribution shifts, the affected population changes, or the model is retrained.

LL 144 requires an audit conducted within the year before each use, so compliance is recurring rather than one-time. Production monitoring should track demographic outcome distributions and alert when statistically significant disparities emerge.