Skip to content

statspai.rlasso

rlasso

Rigorous (data-driven) Lasso — a faithful port of R's hdm package.

Public surface
  • :func:rlasso — rigorous (post-)Lasso with a data-driven, theory- justified penalty (hdm::rlasso).
  • :func:rlasso_effect / :func:rlasso_effects — treatment-effect inference after Lasso-selecting controls (hdm::rlassoEffect(s)).
  • :func:rlasso_iv — instrumental-variables estimation with Lasso selection of instruments and/or controls (hdm::rlassoIV).
  • :class:RlassoRegressor / :class:RlassoClassifier — scikit-learn adapters so the rigorous Lasso can serve as a Double-ML nuisance learner (sp.dml(..., ml_g='rlasso')).

Every estimator is validated to agree numerically with hdm (see tests/reference_parity/test_rlasso_parity.py): the core rlasso matches to machine precision; the IV/effect estimators to ~1e-6.

References

Chernozhukov, V., Hansen, C. and Spindler, M. (2016). "hdm: High-Dimensional Metrics." The R Journal, 8(2), 185-199. [@chernozhukov2016hdm]

RLassoFit dataclass

Bases: ResultProtocolMixin

Result of :func:rlasso — mirrors the fields of an hdm rlasso.

predict

predict(newX: ndarray) -> ndarray

Predict on the original (uncentered) scale — hdm::predict.rlasso.

RLassoLogitFit dataclass

Bases: ResultProtocolMixin

Result of :func:rlassologit — mirrors hdm rlassologit.

predict

predict(newX: ndarray, type: str = 'response') -> ndarray

Predict on newX — type='response' (probabilities) or 'link' (log-odds). Mirrors hdm::predict.rlassologit.

RLassoEffectResult dataclass

Bases: ResultProtocolMixin

Return of :func:rlasso_effect.

RLassoEffectsResult

Bases: Dict[str, RLassoEffectResult]

Effects of several targets, with their joint covariance.

A dict from column name to :class:RLassoEffectResult, as :func:rlasso_effects has always returned, plus what joint inference needs. The estimates are asymptotically jointly normal with covariance Omega / n, Omega[j, l] = E[e_j v_j e_l v_l] / (E[v_j^2] E[v_l^2]), which is what confint(<rlassoEffects>, joint = TRUE) simulates from in hdm [@chernozhukov2016hdm].

to_dict

to_dict() -> Dict[str, Any]

Each target's result, and the joint covariance of the estimates.

vcov

vcov() -> DataFrame

Joint covariance Omega / n of the estimates.

With cluster= the scores are summed within cluster first.

conf_int

conf_int(level: float = 0.95, joint: bool = False, n_draws: int = 100000, seed: Optional[int] = 0) -> DataFrame

Confidence intervals, pointwise or simultaneous.

Parameters:

Name Type Description Default
level float
0.95
joint bool

False: each interval is estimate -/+ z * se with the standard error of the single-target fit. True: a sup-t band, estimate -/+ c * sqrt(diag(vcov)), where c is the level quantile of max_j |Z_j| for Z normal with the correlation matrix of the estimates; all intervals then cover at once with probability level.

False
n_draws int

Draws for the critical value (hdm uses 500).

100_000
seed int or None
0

Returns:

Type Description
DataFrame

Columns estimate, se, lower, upper. The critical value is in .attrs['critical_value'].

RLassoIVResult dataclass

Bases: ResultProtocolMixin

Return of :func:rlasso_iv.

RlassoClassifier

Bases: _SklearnCompatEstimator

Linear-probability classifier backed by the rigorous Lasso.

Fits rlasso to the 0/1 label and exposes clipped predict_proba. Use only when a linear-probability model is acceptable. For a propensity score that is inverted (sp.ipw, sp.aipw, sp.dml(model='irm')) use :class:RlassologitClassifier: a linear prediction below clip is set to clip, so a treated row with extreme covariates gets weight 1 / clip. predict_proba warns (AssumptionWarning) whenever the clip binds.

Examples:

>>> import statspai as sp
>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> X = rng.standard_normal((200, 20))
>>> lin = X[:, 0] - 0.5 * X[:, 1]
>>> d = (rng.uniform(size=200) < 1.0 / (1.0 + np.exp(-lin))).astype(float)
>>> clf = sp.RlassoClassifier().fit(X, d)
>>> clf.predict_proba(X).shape  # columns: P(0), P(1)
(200, 2)

RlassologitClassifier

Bases: _SklearnCompatEstimator

Logistic rigorous-Lasso classifier — a genuine (calibrated) propensity.

Unlike :class:RlassoClassifier (a clipped linear-probability model), this fits rlassologit (the logistic rigorous Lasso, glmnet-aligned) and exposes proper logistic predict_proba. It is the principled sparse nuisance learner for binary treatments / instruments in sp.dml(model='irm'/'iivm', ml_m='rlassologit').

Examples:

>>> import statspai as sp
>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> X = rng.standard_normal((300, 20))
>>> p = 1 / (1 + np.exp(-(X[:, 0] - X[:, 1])))
>>> d = (rng.uniform(size=300) < p).astype(float)
>>> clf = sp.RlassologitClassifier().fit(X, d)
>>> clf.predict_proba(X).shape  # columns: P(0), P(1)
(300, 2)

RlassoRegressor

Bases: _SklearnCompatEstimator

Rigorous (post-)Lasso as a scikit-learn regressor.

Parameters mirror :func:statspai.rlasso.rlasso. Suitable as ml_g / ml_m in sp.dml(model='plr', ...).

Examples:

>>> import statspai as sp
>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> X = rng.standard_normal((100, 20))
>>> beta = np.zeros(20); beta[:3] = [1.0, -1.0, 0.5]
>>> y = X @ beta + 0.5 * rng.standard_normal(100)
>>> est = sp.RlassoRegressor(post=True).fit(X, y)
>>> est.predict(X).shape
(100,)
>>> est.coef_.shape
(20,)

RLassoLogitEffectResult dataclass

Bases: ResultProtocolMixin

Return of :func:rlassologit_effect.

rlasso

rlasso(X: ndarray, y: ndarray, post: bool = True, intercept: bool = True, penalty: Optional[Dict[str, Any]] = None, control: Optional[Dict[str, Any]] = None, colnames: Optional[List[str]] = None, rng: Optional[Generator] = None, cluster: Optional[Any] = None, absorb: Optional[Any] = None) -> RLassoFit

Rigorous Lasso / post-Lasso — a faithful port of hdm::rlasso.

Parameters:

Name Type Description Default
X (n, p) array

Design matrix of candidate covariates (p may exceed n).

required
y (n,) array

Response.

required
post bool

If True, re-estimate the selected support by OLS (post-Lasso).

True
intercept bool

Center X and y and report an intercept on the original scale.

True
penalty dict

Overrides for homoscedastic (True / False / "none"), X.dependent.lambda (bool), c (slack, default 1.1), gamma (default 0.1/log(n)), lambda.start and numSim. Defaults reproduce hdm exactly.

None
control dict

Overrides for numIter (default 15), tol (default 1e-5), threshold (default None) and iteration. iteration is "hdm" (the default without absorb) or "lassopack" (the default with it). "lassopack" follows Stata's rlasso: the first Lasso uses the full penalty (hdm halves it when post=True), the loadings are refreshed from the post-Lasso residuals, at most numIter = 2 Lasso fits are run (maxpsiiter) and the iteration stops early when the residual RMSE moves by less than tol = 1e-4 (tolpsi). The two schemes can stop at different supports because the loading iteration may have more than one fixed point. "lassopack" needs the default heteroskedastic or cluster loadings.

None
colnames list of str

Names for the columns of X (default V1..Vp).

None
rng numpy Generator

Only used when X.dependent.lambda simulation is requested.

None
cluster (n,) array-like

Cluster identifier. Switches to the cluster-Lasso of Belloni, Chernozhukov, Hansen and Kozbur (2016) [@belloni2016inference]: the loading of column j becomes sqrt(sum_g (sum_{i in g} x_ij e_i)^2 / n), which keeps the within-cluster dependence of the scores, and gamma defaults to 0.1 / log(G) with G the number of clusters (the rlasso, cluster() rule of Stata's lassopack). Under positive within-cluster correlation the loadings are larger than the heteroskedastic ones, so fewer variables are selected; ignoring the clustering over-selects. In a fixed-effects panel pass the within-transformed X and y with intercept=False. Requires the default heteroskedastic loadings (homoscedastic=False, no X.dependent.lambda).

None
absorb array - like

Fixed effects to absorb: one identifier per row, a 2-D array with one column per factor, or a list of such arrays. X and y are within-transformed (projected off the dummies of every factor) and the Lasso is run without an intercept, as Stata's rlasso, fe does [@belloni2016inference]; the penalty level still uses all n rows. intercept is ignored, and control defaults to iteration="lassopack". With one factor this reproduces rlasso ..., fe; with two, rlasso ..., fe partial(i.<f2>). Coefficients, residuals and predict are on the within scale.

None

Returns:

Type Description
RLassoFit

Examples:

>>> import statspai as sp
>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> X = rng.standard_normal((100, 20))
>>> beta = np.zeros(20); beta[:3] = [1.0, -1.0, 0.5]
>>> y = X @ beta + 0.5 * rng.standard_normal(100)
>>> fit = sp.rlasso(X, y, post=True)  # rigorous post-Lasso
>>> int(fit.index.sum()) >= 1  # at least one control kept
True
>>> fit.beta.shape
(20,)

rlassologit

rlassologit(X: ndarray, y: ndarray, post: bool = True, intercept: bool = True, penalty: Optional[Dict[str, Any]] = None, control: Optional[Dict[str, Any]] = None, colnames: Optional[List[str]] = None) -> RLassoLogitFit

Logistic rigorous (post-)Lasso — a faithful port of hdm::rlassologit.

Parameters:

Name Type Description Default
X (n, p) array of candidate covariates.
required
y (n,) binary outcome in {0, 1}.
required
post bool

If True, refit the selected support by unpenalized logistic regression (post-Lasso); else keep the glmnet-penalized fit.

True
intercept bool

Include an intercept.

True
penalty dict

Overrides for c (slack; default 1.1 for post=True, else 0.5), gamma (default 0.1/log n) and lambda (raw penalty; bypasses the data-driven level).

None
control dict

threshold — coefficients below it are zeroed (default None).

None
colnames list of str

Column names (default V1..Vp).

None

Returns:

Type Description
RLassoLogitFit

Examples:

>>> import statspai as sp
>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> X = rng.standard_normal((300, 20))
>>> p = 1 / (1 + np.exp(-(X[:, 0] - X[:, 1])))
>>> y = (rng.uniform(size=300) < p).astype(float)
>>> fit = sp.rlassologit(X, y)
>>> fit.n_selected >= 1
True

rlasso_effect

rlasso_effect(x: Union[ndarray, DataFrame, Sequence[str]], y: Union[ndarray, Series, str], d: Union[ndarray, Series, str], method: str = 'partialling out', post: bool = True, I3: Optional[ndarray] = None, data: Optional[DataFrame] = None, penalty: Optional[Dict[str, Any]] = None, control: Optional[Dict[str, Any]] = None, cluster: Optional[Union[ndarray, Series, str]] = None, robust: Optional[bool] = None, absorb: Optional[Any] = None) -> RLassoEffectResult

Effect of d on y after Lasso-selecting controls x.

Faithful port of hdm::rlassoEffect.

Parameters:

Name Type Description Default
x (n, p) controls (array, DataFrame or column names).
required
y outcome and the single target regressor.
required
d outcome and the single target regressor.
required
method ('partialling out', 'double selection')

See the module docstring.

"partialling out"
post bool

Post-Lasso inside the selection steps.

True
I3 bool array

Amelioration set forced into the control set (double-selection only) — hdm's I3 argument.

None
data DataFrame backing string/column-name inputs.
None
penalty dict

Forwarded to :func:statspai.rlasso.rlasso.

None
control dict

Forwarded to :func:statspai.rlasso.rlasso.

None
cluster array - like or str

Cluster identifier (a column name when data is given). Every selection step then uses the cluster-Lasso loadings of :func:statspai.rlasso.rlasso and the standard error is cluster-robust [@belloni2016inference] with no small-sample factor, as in Stata's pdslasso, cluster() (multiply the variance by G / (G - 1) for the regress, cluster() convention). For a fixed-effects panel pass within-transformed x, y and d. hdm has no counterpart.

None
robust bool

Standard error without cluster. None keeps hdm's choice for the method: homoskedastic for "partialling out" (the OLS slope variance of the residual regression), heteroskedasticity-robust for "double selection". True gives the robust variance sum_i v_i^2 e_i^2 / (sum_i v_i^2)^2 for partialling out (no small-sample factor, the cluster formula with one row per cluster); False gives the OLS variance of the final regression for double selection. vce="robust" / vce="iid" set it.

.. versionchanged:: 1.41 vce="robust" used to be accepted and ignored, returning the homoskedastic SE of partialling out.

None
absorb str, list of str, or array-like

Fixed effects to absorb (column names when data is given, or one identifier array per factor). y, d and every control are within-transformed before any Lasso, and the estimator then follows Stata's pdslasso y d (x...), fe [@belloni2016inference] rather than hdm, which has no fixed-effects option: the selection steps use control={"iteration": "lassopack"} unless control says otherwise, the final regressions have no intercept, and no variance carries a small-sample factor. robust=None then means homoskedastic e'e / n for partialling out and HC0 for double selection; robust=True is HC0 for both, robust=False the homoskedastic form for both. Selection matches pdslasso ..., fe robust or fe cluster() (Stata's default homoskedastic penalty is not offered). With several factors the result equals pdslasso ..., fe partial(i.<f2> ...).

None

Returns:

Type Description
RLassoEffectResult

Examples:

>>> import statspai as sp
>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> X = rng.standard_normal((200, 12))  # candidate controls
>>> d = X[:, 0] + rng.standard_normal(200)  # treatment
>>> y = 1.5 * d + X[:, 1] + rng.standard_normal(200)
>>> res = sp.rlasso_effect(X[:, 1:], y, d, method="partialling out")
>>> float(res.se) > 0
True
>>> bool(np.isfinite(res.alpha))  # ~1.5
True

rlasso_effects

rlasso_effects(X: Union[ndarray, DataFrame], y: Union[ndarray, Series], index: Optional[Sequence[int]] = None, method: str = 'partialling out', post: bool = True, data: Optional[DataFrame] = None, penalty: Optional[Dict[str, Any]] = None, control: Optional[Dict[str, Any]] = None, cluster: Optional[Union[ndarray, Series, str]] = None, robust: Optional[bool] = None, absorb: Optional[Any] = None) -> 'RLassoEffectsResult'

Estimate the effect of each targeted column of X on y.

Faithful port of hdm::rlassoEffects: for every target column j in index, treat column j as d and the remaining columns as controls. robust (and vce=) and absorb are passed to :func:rlasso_effect for every target; with absorb each target follows pdslasso ..., fe (see :func:rlasso_effect).

Returns:

Type Description
RLassoEffectsResult

A dict mapping column name -> RLassoEffectResult, with .vcov() (joint covariance of the estimates) and .conf_int(joint=True) (simultaneous sup-t band, hdm's confint(..., joint = TRUE)).

Examples:

>>> import statspai as sp
>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> X = rng.standard_normal((200, 8))
>>> y = X[:, 0] - 0.8 * X[:, 1] + rng.standard_normal(200)
>>> out = sp.rlasso_effects(X, y, index=[0, 1], method="partialling out")
>>> len(out)  # one result per targeted column
2
>>> all(r.se > 0 for r in out.values())
True

Intervals that cover both effects at once are wider than the pointwise ones:

>>> band = out.conf_int(joint=True)
>>> bool(band.attrs["critical_value"] > 1.96)
True

rlasso_iv

rlasso_iv(y: Union[ndarray, Series, str], d: Union[ndarray, Series, str], z: Union[ndarray, DataFrame, Sequence[str]], x: Optional[Union[ndarray, DataFrame, Sequence[str]]] = None, data: Optional[DataFrame] = None, select_Z: bool = True, select_X: bool = True, post: bool = True, intercept: bool = True, penalty: Optional[Dict[str, Any]] = None, control: Optional[Dict[str, Any]] = None, cluster: Optional[Union[ndarray, Series, str]] = None, robust: Optional[bool] = None) -> RLassoIVResult

Instrumental-variables estimation with rigorous-Lasso selection.

A faithful port of hdm::rlassoIV. Estimates the causal effect of an endogenous d on y using (potentially many) instruments z and optional high-dimensional controls x.

Parameters:

Name Type Description Default
y outcome and the endogenous regressor (array, Series or column

name).

required
d outcome and the endogenous regressor (array, Series or column

name).

required
z candidate instruments — ``p_z`` may exceed ``n``.
required
x exogenous controls (optional).
None
data DataFrame backing any string/column-name inputs.
None
select_Z bool

Lasso-select among the instruments.

True
select_X bool

Lasso-select among the controls (partialling-out).

True
post bool

Post-Lasso (OLS refit) inside every selection step.

True
intercept bool

Fit an intercept in every rlasso step (all selection paths) and in the second stage of the select_Z=True, select_X=False path. Set False only for data that are already centred, e.g. within-transformed panels.

True
penalty dict

Forwarded to :func:statspai.rlasso.rlasso (penalty level, loadings, iteration controls).

None
control dict

Forwarded to :func:statspai.rlasso.rlasso (penalty level, loadings, iteration controls).

None
cluster array - like or str

Cluster identifier (a column name when data is given). All selection steps use the cluster-Lasso loadings [@belloni2016inference] and the reported variance is cluster-robust with no small-sample factor (ivlasso's rule). For a fixed-effects panel pass within-transformed variables and intercept=False. With select_Z=True, select_X=False and no controls the point estimate is the post-Lasso IV estimate of Stata's ivlasso, cluster().

None
robust bool

Standard error without cluster. None keeps hdm's choice for each path: homoskedastic for select_Z=False, select_X=True, heteroskedasticity-robust (HC0) for the other three. True / False force the robust / classical 2SLS variance on any path; vce="robust" / vce="iid" set it.

.. versionchanged:: 1.41 vce="robust" used to be accepted and ignored, returning the homoskedastic SE of the select_X-only path.

None

Returns:

Type Description
RLassoIVResult
Notes

With select_Z=True, select_X=False this is the Belloni-Chen- Chernozhukov-Hansen (2012) optimal-instrument estimator that the eminent-domain application made famous. Three paths match hdm::rlassoIV 0.3.2 to ~1e-14. This one departs from hdm::rlassoIVselectZ in three places, all reference bugs:

  • the second stage keeps an intercept when intercept=True and the controls do not already span a constant. hdm drops it, so with a nonzero mean in y or d its slope is inconsistent (truth -2, hdm -0.65 on tests/reference_parity/_fixtures/ rlasso_iv_nonzero_mean.csv). With every instrument selected the estimate is now exactly robust 2SLS;
  • the second stage is solved exactly, by partialling the controls out with a pivoted QR. hdm takes MASS::ginv of Dhat'[d, x], which squares the conditioning of x; on EminentDomain logGDP (x full rank, cond 7e6) the default cutoff discards 34 of 81 singular values and hdm's 0.2274 is a truncated-SVD value that moves with the tolerance. The IV estimate is 0.0154 (SE 0.0141), equal to AER::ivreg with HC0 SEs to 1e-12;
  • the sandwich is A^{-1} Omega A^{-1}' with A = Dhat'[d, x]; hdm transposes A, which changes the SE whenever the first stage leaves some control out (hdm's own tsls(homoscedastic=FALSE) on [dhat, 1, x] gives the corrected value).

With double selection and no control selected for the instrument, the instrument is centred at its own mean; hdm centres it at the grand mean of the control matrix, which leaves the slope unchanged but inflates the SE.

Examples:

>>> import statspai as sp
>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> Z = rng.standard_normal((300, 5))  # candidate instruments
>>> X = rng.standard_normal((300, 10))  # candidate controls
>>> d = Z[:, 0] + 0.5 * X[:, 0] + rng.standard_normal(300)
>>> y = 1.0 * d + X[:, 0] + rng.standard_normal(300)
>>> res = sp.rlasso_iv(y=y, d=d, z=Z, x=X, select_Z=True, select_X=False)
>>> float(res.se[0]) > 0  # res.coef[0] ~ 1.0 (BCH optimal-IV)
True

rlassologit_effect

rlassologit_effect(x: Union[ndarray, DataFrame, Sequence[str]], y: Union[ndarray, Series, str], d: Union[ndarray, Series, str], post: bool = True, I3: Optional[ndarray] = None, data: Optional[DataFrame] = None) -> RLassoLogitEffectResult

Effect of d on a binary y after Lasso-selecting controls x.

Faithful port of hdm::rlassologitEffect.

Parameters:

Name Type Description Default
x (n, p) controls (array, DataFrame or column names).
required
y binary outcome.
required
d the single treatment / target regressor.
required
post bool

Post-Lasso inside the two selection steps.

True
I3 bool array

Amelioration set forced into the control set (hdm's I3).

None
data DataFrame backing string/column-name inputs.
None

Returns:

Type Description
RLassoLogitEffectResult

Examples:

>>> import statspai as sp
>>> import numpy as np
>>> rng = np.random.default_rng(0)
>>> X = rng.standard_normal((300, 10))
>>> d = (rng.standard_normal(300) + X[:, 0] > 0).astype(float)
>>> eta = 0.7 * d + X[:, 1] - 0.5 * X[:, 2]
>>> y = (rng.uniform(size=300) < 1 / (1 + np.exp(-eta))).astype(float)
>>> res = sp.rlassologit_effect(X, y, d)
>>> bool(res.se > 0) and bool(np.isfinite(res.alpha))
True

rlassologit_effects

rlassologit_effects(X: Union[ndarray, DataFrame], y: Union[ndarray, Series], index: Optional[Sequence[int]] = None, post: bool = True, I3: Optional[ndarray] = None, data: Optional[DataFrame] = None) -> Dict[str, RLassoLogitEffectResult]

Logistic high-dimensional effect of each targeted column of X.

Faithful port of hdm::rlassologitEffects: for every target column j in index, treat column j as d and the remaining columns as controls, with a binary outcome y.

Returns:

Type Description
dict

Mapping column name -> RLassoLogitEffectResult.

Examples:

>>> import statspai as sp
>>> import numpy as np
>>> rng = np.random.default_rng(1)
>>> X = rng.standard_normal((300, 8))
>>> eta = 0.8 * X[:, 0] - 0.6 * X[:, 1]
>>> y = (rng.uniform(size=300) < 1 / (1 + np.exp(-eta))).astype(float)
>>> out = sp.rlassologit_effects(X, y, index=[0, 1])
>>> len(out) == 2 and all(r.se > 0 for r in out.values())
True