Research Brief

FairRisk-FDI

FairRisk-FDI extends fairness disagreement analysis into decision-aware deployment risk evaluation for high-stakes AI systems.

Why This Matters

Fairness metrics can produce conflicting conclusions about the same AI system. In sensitive deployment environments, these conflicts are not only reporting issues; they can directly influence whether a system appears deployable, requires mitigation, or should be treated as unsafe.

FairRisk-FDI links fairness disagreement, decision divergence, threshold sensitivity, and domain-specific risk weighting into a structured deployment-risk evaluation framework.

Key Contributions

• Reframes fairness metric disagreement as a deployment-risk problem.
• Connects fairness evaluation to deployment decisions under uncertainty.
• Introduces decision-aware analysis of conflicting fairness outcomes.
• Supports risk-sensitive evaluation across healthcare and facial AI systems.

Deployment Relevance

The framework supports AI assurance by identifying cases where metric-only evaluation may underestimate operational deployment risk. It is especially relevant where different error types carry different consequences across domains, groups, or operational settings.