statspai.fairness

fairness

Counterfactual fairness & algorithmic-bias diagnostics (sp.fairness).

Treats fairness as a causal-inference problem rather than a pure statistics one: an algorithm is counterfactually fair if the prediction would be the same had the protected attribute been different, holding the non-protected causal ancestors fixed.

Core tools

• :func:counterfactual_fairness — Kusner, Loftus, Russell, Silva (2018) Level-2/3 predictor evaluation on a user-supplied SCM.
• :func:orthogonal_to_bias — Chen & Zhu (arXiv:2403.17852v3, 2024) Data pre-processing that removes the part of non-protected features correlated with the protected attribute, via residualization.
• :func:demographic_parity — P(Y_hat=1 | A=a) — A.
• :func:equalized_odds — P(Y_hat=1 | Y=y, A=a) — A.
• :func:fairness_audit — one-shot dashboard combining the metrics above.

References

Kusner, M. J., Loftus, J., Russell, C., & Silva, R. (2018). Counterfactual fairness. NeurIPS.

Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learning. NeurIPS.

Chen, S., & Zhu, S. (2024). Counterfactual Fairness Through Orthogonal to Bias. arXiv:2403.17852v3.

FairnessResult dataclass

Single fairness diagnostic.

FairnessAudit dataclass

One-shot dashboard of fairness diagnostics.

EvidenceWithoutInjusticeResult dataclass

Bases: FairnessResult

Extended fairness result with bootstrap CI and per-alt statistics.

Inherits the standard metric/value/per_group/threshold/passes/notes fields from :class:FairnessResult and adds bootstrap-inference artefacts specific to Kwak-Pleasants 2025.

counterfactual_fairness

counterfactual_fairness(data: DataFrame, predictor: Callable[[DataFrame], ndarray], *, protected: str, scm_intervention: Callable[[DataFrame, Any], DataFrame], alternative_values: Optional[Sequence[Any]] = None, threshold: float = 0.05) -> FairnessResult

Kusner-Loftus-Russell-Silva counterfactual-fairness test.

For each unit, compare the factual prediction with the counterfactual prediction obtained under a user-supplied SCM intervention that sets the protected attribute A to alternative values.

Parameters:

Name	Type	Description	Default
data	DataFrame	Observed covariates (including the protected attribute).	required
predictor	Callable(DataFrame) -> ndarray	Deployed predictor. Must accept a DataFrame and return an array of the same length.	required
protected	str	Protected attribute column — its counterfactual is constructed by scm_intervention.	required
scm_intervention	Callable(DataFrame, value) -> DataFrame	User-supplied causal model. Given the original data and an alternative value of A, returns a modified DataFrame representing the counterfactual world (with downstream descendants updated under the intervention). The caller owns the SCM.	required
alternative_values	sequence	Values of A to intervene on. Defaults to all unique values of data[protected] other than the observed one.	None
threshold	float		0.05

Returns:

Type	Description
FairnessResult	value is the average absolute counterfactual change, mean_i max_{a'} | f(X_i^{a'}) - f(X_i^{a_i}) |.

Notes

This is a Level-3 (counterfactual) fairness test and is only as credible as the SCM the user supplies. A DAG + structural equations must be specified outside this function — we just wrap the mechanics.

References

Kusner, Loftus, Russell, Silva (2018).

orthogonal_to_bias

orthogonal_to_bias(data: DataFrame, *, features: Sequence[str], protected: str, method: str = 'residualize') -> DataFrame

OB preprocessing — remove the part of features correlated with A.

For each feature X_j, regress X_j ~ A (one-hot A if multi-valued) and replace the feature by its residual. The residualized features are by construction uncorrelated with A in-sample. Training a predictor on the residualized features is a simple relaxation of counterfactual fairness that requires no explicit SCM.

Parameters:

Name	Type	Description	Default
data	DataFrame		required
features	sequence of str	Numeric columns to residualize.	required
protected	str	Protected attribute column (numeric or categorical).	required
method	'residualize'		'residualize'

Returns:

Type	Description
DataFrame	Copy of data with the features columns replaced by residualized versions. Other columns (including protected) unchanged.

References

Chen & Zhu (arXiv:2403.17852v3, 2024).

demographic_parity

demographic_parity(data: DataFrame, *, predictions: str, protected: str, threshold: float = 0.1) -> FairnessResult

Demographic-parity gap: max_{a,b} | P(Y_hat=1 | A=a) − P(Y_hat=1 | A=b) |.

Parameters:

Name	Type	Description	Default
data	DataFrame		required
predictions	str	Column of binary classifier outputs (0/1).	required
protected	str	Column containing the protected attribute A. May be multi-valued.	required
threshold	float	A gap below this value counts as a "pass". Follows the 80%-rule of thumb (EEOC disparate-impact guideline).	0.1

Notes

Demographic parity is the weakest fairness criterion — it ignores ground-truth labels and can be trivially satisfied by a random classifier. Use together with :func:equalized_odds.

equalized_odds

equalized_odds(data: DataFrame, *, predictions: str, labels: str, protected: str, threshold: float = 0.1) -> FairnessResult

Hardt-Price-Srebro equalized-odds gap.

Returns the larger of the TPR gap and FPR gap across groups:

gap = max( max |TPR_a − TPR_b|, max |FPR_a − FPR_b| )

Parameters:

Name	Type	Description	Default
data	DataFrame		required
predictions	str	Binary classifier output.	required
labels	str	Binary ground-truth outcome.	required
protected	str	Protected attribute column.	required
threshold	float		0.1

fairness_audit

fairness_audit(data: DataFrame, *, predictions: str, protected: str, labels: Optional[str] = None, predictor: Optional[Callable[[DataFrame], ndarray]] = None, scm_intervention: Optional[Callable[[DataFrame, Any], DataFrame]] = None, alternative_values: Optional[Sequence[Any]] = None, threshold: float = 0.1) -> FairnessAudit

Run all applicable fairness diagnostics on a classifier's output.

Parameters:

Name	Type	Description	Default
data	DataFrame		required
predictions	str	Binary classifier output column.	required
protected	str		required
labels	str	If present, adds equalized-odds diagnostic.	None
predictor	Optional[Callable[[DataFrame], ndarray]]	If predictor and scm_intervention are both supplied, also runs counterfactual-fairness.	None
scm_intervention	Optional[Callable[[DataFrame], ndarray]]	If predictor and scm_intervention are both supplied, also runs counterfactual-fairness.	None
alternative_values	Optional[Callable[[DataFrame], ndarray]]	If predictor and scm_intervention are both supplied, also runs counterfactual-fairness.	None

evidence_without_injustice

evidence_without_injustice(data: DataFrame, predictor: Callable[[DataFrame], ndarray], *, protected: str, admissible_features: Sequence[str], scm_intervention: Callable[[DataFrame, Any], DataFrame], alternative_values: Optional[Sequence[Any]] = None, threshold: float = 0.05, alpha: float = 0.05, n_boot: int = 500, random_state: Optional[int] = None) -> EvidenceWithoutInjusticeResult

Kwak-Pleasants (2025) evidence-without-injustice fairness test.

Parameters:

Name	Type	Description	Default
data	DataFrame	Observed covariates — must include protected and all admissible_features.	required
predictor	Callable(DataFrame) -> ndarray	Deployed model. Called with (a) the original data and (b) the counterfactual data after intervention + admissibility freezing.	required
protected	str	Name of the protected-attribute column.	required
admissible_features	sequence of str	Feature columns whose factual values are preserved in the counterfactual world — these encode the "admissible evidence" that the algorithm may legitimately use even if correlated with A. Pass [] to recover classical counterfactual fairness.	required
scm_intervention	Callable(DataFrame, value) -> DataFrame	User-supplied SCM. Returns a DataFrame of the same shape with protected (and all non-admissible descendants) updated under do(A=value). The admissibility freeze is enforced after calling this function — admissible columns are overwritten with their factual values.	required
alternative_values	sequence	Values of A to intervene on. Defaults to all levels other than the unit's factual value.	None
threshold	float	Practical-significance threshold on T. passes = True iff ci[1] < threshold.	0.05
alpha	float	Significance level for the bootstrap CI.	0.05
n_boot	int		500
random_state	int		None

Returns:

Type	Description
EvidenceWithoutInjusticeResult	Standard :class:FairnessResult fields plus ci, pvalue, n_boot, and admissible_features.

Examples:

>>> import numpy as np, pandas as pd, statspai as sp
>>> rng = np.random.default_rng(0)
>>> n = 500
>>> A = rng.integers(0, 2, n)
>>> credit = 600 + 100 * A + rng.normal(0, 30, n)  # correlated w/ A
>>> race_noise = 0.0 * A  # zero direct effect -> fair
>>> df = pd.DataFrame({'A': A, 'credit': credit, 'noise': rng.normal(size=n)})
>>> def predictor(d): return 1 / (1 + np.exp(-(d['credit'] / 100 - 6)))
>>> def intervene(d, a_alt):
...     out = d.copy(); out['A'] = a_alt
...     out['credit'] = 600 + 100 * a_alt + (d['credit'] - (600 + 100 * d['A']))
...     return out
>>> res = sp.fairness.evidence_without_injustice(
...     df, predictor, protected='A',
...     admissible_features=['credit'],
...     scm_intervention=intervene,
...     n_boot=200, random_state=0,
... )
>>> bool(res.passes)
True

Notes

The bootstrap is a paired nonparametric bootstrap over the units: for each replicate we resample rows with replacement and recompute the test statistic. This accounts for predictor-randomness only via plug-in (the predictor is treated as fixed). For full accounting of predictor uncertainty, pass in a Bayesian posterior predictor or wrap the predictor in its own bootstrap at a higher level.