Outcome-Capped Fairness: A Ceiling-Constrained Measurement Framework for AI Systems

Abstract

We present Parity, a fairness measurement framework in which outcome disparity acts as a hard mathematical ceiling on procedural fairness scores. Using the EEOC four-fifths rule as a continuous measure rather than a binary threshold, Parity ensures that an agent with perfect treatment consistency but disparate outcomes cannot score above its outcome disparity level. The framework integrates Jensen-Shannon divergence for treatment analysis, causal inference for proxy variable detection, and the Theil index for between-group economic equity decomposition.

Context

Fairness measurement in AI systems has bifurcated into two largely disconnected streams ^[1]. The first stream, procedural fairness, examines whether the AI system treats individuals consistently across protected categories. This includes analyses of whether the system uses different decision rules for different groups, whether similar inputs produce similar outputs regardless of group membership, and whether the system's internal representations encode protected attributes that should not influence decisions. The second stream, outcome fairness, examines whether the AI system produces equitable results across protected categories, measured through statistical parity of outcomes such as approval rates, score distributions, and economic impacts.

The disconnect between these streams creates a measurement gap that existing frameworks fail to address ^[2]. A lending algorithm that applies identical decision rules to all applicants (high procedural fairness) but produces a 3:1 approval rate disparity between demographic groups (low outcome fairness) will receive a high score on procedural metrics and a low score on outcome metrics. If these metrics are reported separately, the system appears simultaneously fair and unfair depending on which metric is consulted. If they are averaged, the composite conceals the outcome disparity behind procedural excellence, producing a moderate score that misrepresents the system's actual fairness impact.

The legal and regulatory landscape strongly suggests that outcome fairness should constrain procedural fairness assessments rather than being averaged with them. The EEOC's four-fifths rule, established in the 1978 Uniform Guidelines on Employee Selection Procedures ^[3], provides the canonical example: a selection procedure is presumed to have adverse impact if the selection rate for any protected group is less than four-fifths of the rate for the group with the highest rate. This is an outcome-based standard that applies regardless of procedural consistency. A perfectly consistent procedure that produces disparate outcomes is presumptively discriminatory under this framework ^[4]. The legal standard treats outcomes as a ceiling on permissible procedural fairness claims.

The Parity framework operationalizes this legal intuition mathematically. Rather than treating procedural and outcome fairness as independent dimensions to be aggregated, Parity defines outcome fairness as a hard ceiling on the composite score. An agent with perfect procedural scores across all treatment analysis dimensions but significant outcome disparity receives a composite score capped at the level determined by the outcome disparity. This ceiling-constrained architecture ensures that Parity scores cannot be inflated by procedural excellence when outcomes are disparate, which is precisely the evaluation logic that anti-discrimination law applies.

Architecture

The Parity framework evaluates fairness across four procedural dimensions and one outcome dimension. The outcome dimension establishes the ceiling; the procedural dimensions are aggregated beneath it. The outcome dimension operationalizes the EEOC four-fifths rule as a continuous measure rather than a binary threshold ^[3]. Traditional application of the four-fifths rule produces a binary result: the selection rate ratio either exceeds 0.8 or it does not. Parity transforms this into a continuous score by computing the minimum selection rate ratio across all pairwise group comparisons and mapping it to the 0-100 scale. A ratio of 1.0 (perfect parity) maps to 100; a ratio of 0.8 (the four-fifths threshold) maps to approximately 65; a ratio of 0.5 maps to approximately 30; a ratio approaching 0 maps to near 0.

The first procedural dimension, treatment consistency, measures whether the agent applies consistent decision logic across protected groups using the Jensen-Shannon divergence (JSD) ^[5]. For each protected group, Parity computes the distribution of agent outputs conditional on equivalent inputs and calculates the JSD between each pair of group-conditional output distributions. The JSD is preferred over the Kullback-Leibler divergence because it is symmetric and bounded between 0 and 1, producing well-behaved scores that are interpretable as the information-theoretic distance between group treatment patterns. A JSD of 0 indicates identical treatment distributions; a JSD approaching 1 indicates maximally different treatment.

The second procedural dimension, proxy variable detection, uses causal inference techniques ^[6] to identify variables in the agent's decision process that serve as proxies for protected attributes ^[7]. An agent that does not directly use race in lending decisions but heavily weights ZIP code, which is strongly correlated with race, is using a proxy variable that produces the same discriminatory outcomes as direct use of the protected attribute. Parity employs a causal graph analysis that identifies paths from protected attributes to decision outcomes that pass through non-protected variables, quantifying the indirect effect of protected attributes on decisions through proxy pathways.

The third procedural dimension, explanation consistency, measures whether the agent provides consistent explanations for similar decisions across protected groups. An agent that explains an application denial as "insufficient income" for applicants in one group but as "high risk profile" for applicants in another group, when the underlying financial characteristics are equivalent, exhibits explanation inconsistency that may indicate differential treatment. This dimension uses semantic similarity analysis on agent-generated explanations, comparing explanation distributions across groups conditional on equivalent input profiles.

The fourth procedural dimension, between-group economic equity, uses the Theil index ^[8] to decompose economic outcomes into within-group and between-group components. The Theil index is selected because it is perfectly decomposable: the total inequality in a population can be expressed as the sum of within-group inequality and between-group inequality, with no residual term. The between-group component measures the extent to which the agent's decisions create or perpetuate economic disparities between protected groups, distinct from the overall inequality that may reflect pre-existing economic conditions. This decomposition enables Parity to measure the agent's marginal contribution to between-group inequality rather than holding the agent responsible for pre-existing disparities.

Specification

The composite Parity score is computed through a ceiling-constrained aggregation that implements the outcome-over-procedure hierarchy. Let O represent the outcome disparity score on the 0-100 scale, and let P_1, P_2, P_3, P_4 represent the four procedural dimension scores. The procedural composite is computed as P_composite = weighted_arithmetic_mean(P_1, P_2, P_3, P_4), using the arithmetic mean because the procedural dimensions are genuinely compensatory with respect to each other. The final Parity score is then min(O, P_composite), ensuring that the outcome disparity score acts as a hard ceiling that no amount of procedural excellence can breach.

The ceiling constraint produces distinctive score patterns that communicate meaningful fairness information. An agent with O = 40 and P_composite = 90 receives a Parity score of 40, clearly communicating that outcome disparity is the binding constraint and that procedural improvements cannot improve the composite. An agent with O = 85 and P_composite = 60 receives a Parity score of 60, communicating that procedural improvements can increase the composite up to the ceiling of 85. An agent with O = 95 and P_composite = 92 receives a Parity score of 92, communicating that both procedural and outcome fairness are strong and the ceiling is not binding. These patterns enable users and regulators to immediately identify whether an agent's fairness challenges are primarily outcome-based or procedural.

The Jensen-Shannon divergence computation for treatment consistency requires careful specification to handle the high-dimensional output spaces typical of AI agents. For agents with continuous output spaces, Parity discretizes outputs using adaptive binning that preserves the distributional characteristics relevant to fairness analysis. The bin boundaries are determined using Bayesian blocks ^[9], an algorithm that optimizes the trade-off between bin resolution and statistical significance, ensuring that the JSD computation is robust to both sparse data and distributional complexity. For agents with discrete output spaces (approve/deny, classification labels), the JSD is computed directly on the empirical distributions without discretization.

The proxy variable detection mechanism employs the mediation analysis framework from causal inference ^[10]. For each non-protected variable V in the agent's feature space, Parity computes the natural indirect effect (NIE) of each protected attribute A on the decision outcome Y through V. The NIE quantifies how much of the total effect of A on Y is transmitted through V, providing a direct measure of V's role as a proxy. Variables with NIE exceeding a significance threshold are flagged as proxies, and the aggregate proxy influence is incorporated into the proxy detection dimension score. The threshold is calibrated using permutation testing to control the false discovery rate ^[11], ensuring that variables are not spuriously identified as proxies due to sample noise.

The Theil index decomposition for between-group economic equity requires specification of the economic outcome variable and the group partition ^[8]. Parity supports multiple economic outcome variables, including monetary outcomes (loan amounts, pricing), opportunity outcomes (approval rates, access levels), and composite economic outcomes (expected lifetime value). The group partition is defined by the intersection of protected attributes, and the Theil index is decomposed into within-group and between-group components for each partition. The between-group component, normalized to the 0-100 scale, constitutes the economic equity dimension score.

Applications

Lending represents the most mature application domain for the Parity framework, with well-established legal standards and extensive empirical data on disparate impact ^[12]. A lending agent that uses an ML model to approve or deny credit applications can be evaluated by computing approval rate ratios across racial, gender, and age groups for the outcome ceiling; Jensen-Shannon divergence of approval probabilities conditional on equivalent creditworthiness profiles for treatment consistency; natural indirect effects of protected attributes through ZIP code, education institution, and employer variables for proxy detection; and Theil index decomposition of loan amount distributions for economic equity. In our testing with simulated lending agents, Parity scores ranged from 18 for agents with severe disparate impact and active proxy usage to 88 for agents with balanced outcomes and consistent treatment.

Hiring presents unique challenges because the outcome variable is multidimensional: an agent may influence resume screening, interview scheduling, offer generation, and compensation setting, each with its own fairness implications. Parity addresses this by computing separate outcome ceilings for each decision stage and applying the minimum ceiling across stages. This approach captures the cascading nature of hiring disparity, where small biases at each stage compound into large outcome disparities across the full pipeline ^[13]. A hiring agent that is slightly biased at each of five stages may appear moderately fair at each individual stage but receives a low Parity score that reflects the compounded pipeline effect.

Insurance pricing provides an application where the distinction between legitimate actuarial factors and proxy variables is particularly contentious. Age is a protected attribute in many jurisdictions but is also a legitimate actuarial factor for health and life insurance. Parity handles this through the proxy detection mechanism's ability to distinguish between direct use of a protected attribute, which may be legally permitted in specific insurance contexts, and indirect use through proxy variables, which typically is not. The framework does not make the legal determination of whether direct attribute use is permitted; it measures and reports the degree to which protected attributes influence outcomes, leaving the legal interpretation to the regulatory context.

The cross-domain validation reveals that the ceiling constraint is binding in the majority of real-world applications. In our testing across lending, hiring, and insurance domains, the outcome ceiling was the binding constraint for 67% of agents, meaning that most agents' fairness scores were limited by outcome disparity rather than procedural treatment. This finding has significant implications for AI fairness improvement strategies: the majority of agents would benefit more from outcome-focused interventions than from procedural-focused interventions. Organizations that invest primarily in procedural fairness, which is the default approach in the current AI ethics landscape, may find that their Parity scores do not improve because the outcome ceiling remains the binding constraint.

The Parity framework also enables longitudinal fairness monitoring that tracks both the outcome ceiling and the procedural composite over time. An agent whose outcome ceiling is rising while its procedural composite is stable represents a system where external conditions are improving. An agent whose outcome ceiling is stable while its procedural composite is falling represents a system that is degrading procedurally, which may foreshadow future outcome disparity as procedural biases accumulate in outcome distributions. This temporal analysis transforms fairness measurement from a point-in-time compliance assessment into a dynamic monitoring capability that can identify emerging fairness risks before they manifest as outcome disparities.