`statspai.causal_text`¶

causal_text ¶

Causal inference with text data — MVP (P1-B, v1.6 experimental).

Two methods to start the family:

:func:text_treatment_effect — Veitch-Wang-Blei (2020) text-as- treatment via embedding adjustment. Estimates the ATE of a user-supplied treatment indicator while controlling for the confounding variation captured in the text embedding.
:func:llm_annotator_correct — Egami-Hinck-Stewart-Wei (2024) measurement-error correction for LLM-derived treatment labels. Given a small validation subset where both LLM and human labels exist, debias the downstream regression coefficient via Hausman- style correction.

Status

Experimental. Both estimators ship with conservative defaults and validation-set-estimated noise parameters; consume them as a starting point, not a final answer. Future versions will add text-as-confounder (Roberts-Stewart-Nielsen) and text-as-outcome (Egami et al. 2018) methods alongside topic-model integration.

References

Veitch, V., Sridhar, D., & Blei, D. M. (2019). "Adapting text embeddings for causal inference." UAI. arXiv:1905.12741.
Egami, N., Hinck, M., Stewart, B., & Wei, H. (2023). "Using imperfect surrogates for downstream inference: Design-based supervised learning for social science applications of large language models." NeurIPS. arXiv:2306.04746.

LLMAnnotatorResult ¶

Bases: CausalResult

Output of :func:llm_annotator_correct.

Inherits the agent-native CausalResult API. Adds annotator-specific fields naive_estimate, correction_factor, and annotator_diagnostics (false-positive / false-negative rates, validation-set size, agreement rate, optional confusion matrix and inflation factor) on the instance.

Examples:

Normally returned by sp.llm_annotator_correct (which calls an LLM annotator); the result object itself is a plain CausalResult subclass and can be inspected directly:

>>> import statspai as sp
>>> res = sp.LLMAnnotatorResult(
...     method='llm_annotator_correct', estimand='ATE',
...     estimate=0.42, se=0.08, pvalue=0.001, ci=(0.26, 0.58),
...     alpha=0.05, n_obs=500, naive_estimate=0.30, naive_se=0.06,
...     correction_factor=1.4,
...     annotator_diagnostics={'agreement': 0.9, 'n_validation': 100},
... )
>>> round(float(res.estimate), 2)
0.42
>>> isinstance(res, sp.CausalResult)
True

TextTreatmentResult ¶

Bases: CausalResult

ATE result for text-as-treatment estimation.

Subclasses :class:CausalResult so it inherits .tidy(), .to_latex(), .cite(), and the agent-native .to_dict() when P0's additions are present. Exposes embedding-specific metadata on the instance and inside model_info['text_diagnostics'] so agents can introspect what happened.

Examples:

>>> import numpy as np
>>> import pandas as pd
>>> import statspai as sp
>>> rng = np.random.default_rng(0)
>>> n = 200
>>> docs = ['good service' if rng.random() > 0.5 else 'late delivery'
...         for _ in range(n)]
>>> treat = rng.binomial(1, 0.5, size=n)
>>> y = 2.0 * treat + rng.normal(size=n)
>>> df = pd.DataFrame({'review': docs, 'treat': treat, 'y': y})
>>> res = sp.text_treatment_effect(
...     df, text_col='review', outcome='y', treatment='treat',
...     n_components=8, seed=0,
... )
>>> isinstance(res, sp.TextTreatmentResult)
True
>>> res.estimand
'ATE'
>>> res.embedding_dim
8

llm_annotator_correct ¶

llm_annotator_correct(*, annotations_llm: Series, outcome: Series, annotations_human: Optional[Series] = None, covariates: Optional[DataFrame] = None, method: str = 'auto', bootstrap: bool = False, n_bootstrap: int = 500, bootstrap_seed: Optional[int] = None, alpha: float = 0.05) -> LLMAnnotatorResult

Correct a downstream causal coefficient for LLM annotation noise.

Implements three correction paths sharing the same API:

Binary T — β_corrected = β_obs / λ with λ = P(T_true=1 | T_obs=1) - P(T_true=1 | T_obs=0) estimated on the validation rows.
Multi-class T (K ≥ 3) — inverse-confusion-matrix correction. The treatment is dummy-encoded with the smallest label as reference; per-class contrasts are recovered via a linear transformation built from the validation-set Bayes posterior Q[i, j] = P(T_true=i | T_obs=j).
Continuous score — regression calibration. The calibration model S_true ~ S_llm + X is fitted on the human-audited rows, the calibrated score is imputed for every row, and the outcome OLS is re-run on the imputed score. With no covariates this is exactly the classical reliability-ratio correction β_corrected = β_obs / λ with λ = Cov(S_llm, S_true) / Var(S_llm).

Parameters:

Name	Type	Description	Default
`annotations_llm`	`Series`	LLM-derived annotation for every row. Binary / multi-class (numeric class labels) or a continuous score (e.g. sentiment in `[-1, 1]`).	required
`outcome`	`Series`	Outcome variable.	required
`annotations_human`	`Series`	Human annotation; `NaN` where unavailable. At least 30 rows with both LLM and human labels are required; for discrete paths every true class must appear at least once.	`None`
`covariates`	`DataFrame`	Additional control variables for the OLS regression.	`None`
`method`	`('auto', 'hausman', 'reliability')`	Correction method. `'auto'` routes integer-like labels with few distinct values to the discrete Hausman / confusion-matrix path and anything else to the continuous regression-calibration path. `'hausman'` forces the discrete path; `'reliability'` forces the continuous path.	`'auto'`
`bootstrap`	`bool`	If True, run a bias-corrected percentile bootstrap that resamples the full sample (validation rows + unlabeled rows jointly) `n_bootstrap` times. CIs / SE on the result reflect validation-set sampling uncertainty; the first-order versions ship in `model_info['first_order_se']` and `['first_order_ci']`.	`False`
`n_bootstrap`	`int`	Bootstrap replicates.	`500`
`bootstrap_seed`	`int`	Seed for the bootstrap RNG (NumPy `default_rng`).	`None`
`alpha`	`float`	CI level (1 - alpha confidence).	`0.05`

Returns:

Type	Description
`LLMAnnotatorResult`

Examples:

>>> import statspai as sp, pandas as pd, numpy as np
>>> n, n_val = 1000, 100
>>> rng = np.random.default_rng(0)
>>> T_true = (rng.random(n) > 0.5).astype(int)
>>> noise = (rng.random(n) < 0.15).astype(int)
>>> T_llm = (T_true ^ noise).astype(int)            # 15% misclass.
>>> y = 1.0 * T_true + rng.standard_normal(n)        # true ATE 1.0
>>> human = pd.Series([T_true[i] if i < n_val else np.nan
...                    for i in range(n)])
>>> r = sp.llm_annotator_correct(
...     annotations_llm=pd.Series(T_llm),
...     annotations_human=human,
...     outcome=pd.Series(y),
... )
>>> isinstance(r, sp.LLMAnnotatorResult)
True
>>> bool(0.7 < r.estimate < 1.3)    # corrected toward the true ATE 1.0
True

Continuous AI scores (e.g. LLM sentiment) with a human-audited subsample use the regression-calibration path automatically:

>>> s_true = rng.uniform(-1, 1, n)
>>> s_ai = np.clip(s_true + rng.normal(0, 0.5, n), -1, 1)
>>> sales = 100 + 30 * s_true + rng.normal(0, 6, n)
>>> human_s = pd.Series([s_true[i] if i < n_val else np.nan
...                      for i in range(n)])
>>> rc = sp.llm_annotator_correct(
...     annotations_llm=pd.Series(s_ai),
...     annotations_human=human_s,
...     outcome=pd.Series(sales),
... )
>>> rc.estimand
'slope'
>>> bool(abs(rc.estimate - 30) < abs(rc.naive_estimate - 30))
True

References

Egami, Hinck, Stewart & Wei (NeurIPS 2023) — arXiv:2306.04746. [@egami2023imperfect] Hausman, Abrevaya & Scott-Morton (J. Econometrics 1998). [@hausman1998misclassification] Fuller (1987), Measurement Error Models, Wiley — doi:10.1002/9780470316665. [@fuller1987measurement] Carroll, Ruppert, Stefanski & Crainiceanu (2006), Measurement Error in Nonlinear Models (2nd ed.), Chapman & Hall/CRC — doi:10.1201/9781420010138. [@carroll2006measurement]

text_treatment_effect ¶

text_treatment_effect(data: DataFrame, *, text_col: str, outcome: str, treatment: str, covariates: Optional[List[str]] = None, embedder: Union[str, Callable] = 'hash', n_components: int = 20, seed: int = 0, alpha: float = 0.05) -> TextTreatmentResult

Estimate the ATE of a text-derived treatment via embedding adjustment.

Parameters:

Name	Type	Description	Default
`data`	`DataFrame`		required
`text_col`	`str`	Column containing the document text (string).	required
`outcome`	`str`	Outcome column (numeric).	required
`treatment`	`str`	Treatment column (binary or continuous). This is the variable whose coefficient is interpreted as the ATE.	required
`covariates`	`list of str`	Additional non-text covariates to include in the adjustment regression.	`None`
`embedder`	`('hash', 'sbert')`	Text embedder. See :func:`statspai.causal_text._common.embed_texts`.	`'hash'`
`n_components`	`int`	Embedding dimensionality (for 'hash'); ignored for 'sbert'.	`20`
`seed`	`int`		`0`
`alpha`	`float`	CI level (1 - alpha confidence).	`0.05`

Returns:

Type	Description
`TextTreatmentResult`

Notes

The 'hash' embedder drops tokens into n_components buckets, and two tokens that share a bucket cannot be told apart by the adjustment. When the confounding runs through such tokens, part of it stays in the estimate and the standard error does not account for it. On a design with an eight-word vocabulary, a text-borne confounder and a true effect of 1.0, 20 buckets gave a mean of 1.08 with 57% coverage of the nominal 95% interval; 256 buckets gave 1.02 and every interval covered. Raise n_components well above the vocabulary that matters, or pass a dense embedder.

Examples:

>>> import statspai as sp, numpy as np, pandas as pd
>>> rng = np.random.default_rng(0)
>>> df = pd.DataFrame({
...     "text": ["great product love it"] * 15 + ["terrible buggy slow"] * 15,
...     "treatment": [1] * 15 + [0] * 15,
...     "outcome": list(rng.normal(4.0, 0.3, 15)) + list(rng.normal(2.0, 0.3, 15)),
... })
>>> r = sp.text_treatment_effect(df, text_col="text", outcome="outcome",
...                              treatment="treatment", n_components=2)
>>> r.estimand, r.embedding_dim
('ATE', 2)
>>> bool(np.isfinite(r.estimate) and r.se > 0)
True

statspai.causal_text¶

causal_text ¶

LLMAnnotatorResult ¶

TextTreatmentResult ¶

llm_annotator_correct ¶

text_treatment_effect ¶

`statspai.causal_text`¶