statspai.bartik

bartik

Shift-Share (Bartik) Instrumental Variables for StatsPAI.

Constructs Bartik instruments from industry shares and national shocks, with diagnostics for instrument validity following Goldsmith-Pinkham, Sorkin, and Swift (2020) and Borusyak, Hull, and Jaravel (2022).

References

Goldsmith-Pinkham, P., Sorkin, I., and Swift, H. (2020). "Bartik Instruments: What, When, Why, and How." American Economic Review, 110(8), 2586-2624. [@goldsmithpinkham2020bartik]

Borusyak, K., Hull, P., and Jaravel, X. (2022). "Quasi-Experimental Shift-Share Research Designs." Review of Economic Studies, 89(1), 181-213. [@borusyak2022quasi]

ShiftSharePoliticalPanelResult dataclass

Bases: ResultProtocolMixin

Structured output of :func:shift_share_political_panel.

Attributes:

Name	Type	Description
estimate	float	Pooled 2SLS coefficient on endog.
se	float	SE of the requested cluster type (unit-clustered CR0 by default; with cluster='shock' the AKM SE, also stored in diagnostics['akm_se']).
ci	tuple	(lower, upper) at alpha.
per_period	DataFrame	Per-period cross-sectional estimates (one row per time), useful for event-study-style dynamic effects.
rotemberg_panel	DataFrame	Rotemberg weights aggregated across periods (industries × stats).
share_balance	DataFrame
n_units	int
n_periods	int
n_industries	int
method	str
diagnostics	dict	Estimator-side diagnostics: fe (mode), cluster, akm_se, akm0_ci, n_obs, balanced, first_stage_F (homoskedastic F of the excluded instrument, df net of the absorbed FE; fixest::fitstat(, "ivf1")).
model_info	dict	Output-layer metadata: model_type, method, fixed_effects (column-name list as "unit+time"), cluster. Consumed by :func:statspai.regtable to render per-FE / cluster rows automatically.

Examples:

>>> import statspai as sp
>>> import numpy as np
>>> import pandas as pd
>>> rng = np.random.default_rng(0)
>>> units, times, inds = list(range(30)), [0, 1, 2, 3], list("AB")
>>> shares = pd.DataFrame(rng.dirichlet(np.ones(2), size=len(units)),
...                       index=units, columns=inds)
>>> shocks = pd.DataFrame(
...     rng.normal(size=(len(times), 2)), index=times, columns=inds,
... )
>>> rows = []
>>> for i in units:
...     for t in times:
...         b = float((shares.loc[i] * shocks.loc[t]).sum())
...         x = b + rng.normal(scale=0.1)
...         rows.append({"u": i, "t": t, "y": 0.3 * x, "x": x})
>>> df = pd.DataFrame(rows)
>>> res = sp.shift_share_political_panel(
...     df, unit="u", time="t", outcome="y", endog="x",
...     shares=shares, shocks=shocks,
... )
>>> bool(np.isfinite(res.estimate))
True

ShiftSharePoliticalResult dataclass

Bases: ResultProtocolMixin

Structured output of :func:shift_share_political.

Wraps a standard :class:CausalResult (2SLS point estimate, HC1 SE) plus the AKM shock-level SE (diagnostics), Rotemberg weights and a share-balance table.

Examples:

>>> import statspai as sp
>>> import numpy as np
>>> import pandas as pd
>>> rng = np.random.default_rng(0)
>>> units, inds = range(20), [f"I{k}" for k in range(5)]
>>> shares = pd.DataFrame(rng.dirichlet(np.ones(5), size=len(units)),
...                       index=list(units), columns=inds)
>>> shocks = pd.Series(rng.normal(size=5), index=inds)
>>> rows = []
>>> for i in units:
...     dx = float((shares.loc[i] * shocks).sum()) + rng.normal(scale=0.1)
...     rows.append({"unit": i, "time": 0, "y": 0.0, "x": 0.0})
...     rows.append({"unit": i, "time": 1, "y": 0.4 * dx, "x": dx})
>>> df = pd.DataFrame(rows)
>>> res = sp.shift_share_political(
...     df, unit="unit", time="time", outcome="y", endog="x",
...     shares=shares, shocks=shocks,
... )
>>> bool(np.isfinite(res.estimate))
True

BartikIV

Bartik Shift-Share IV estimator.

Constructs the shift-share instrument B_i = sum_k s_ik * g_k from a region-by-industry share matrix and an industry shock vector, then runs 2SLS. The convenience wrapper :func:bartik builds this object and calls :meth:fit; use the class directly when you want to hold onto the estimator instance.

Examples:

>>> import statspai as sp
>>> import numpy as np, pandas as pd
>>> from statspai.bartik.shift_share import BartikIV
>>> rng = np.random.default_rng(0)
>>> n, K = 60, 4
>>> shares = pd.DataFrame(
...     rng.dirichlet(np.ones(K), size=n),
...     columns=[f"ind{k}" for k in range(K)],
... )
>>> shocks = pd.Series(rng.normal(0.05, 0.02, K), index=shares.columns)
>>> emp = shares.values @ shocks.values + rng.normal(0, 0.01, n)
>>> wage = 1.5 * emp + rng.normal(0, 0.02, n)
>>> df = pd.DataFrame({"wage_growth": wage, "emp_growth": emp})
>>> est = BartikIV(df, y="wage_growth", endog="emp_growth",
...                shares=shares, shocks=shocks, leave_one_out=False)
>>> res = est.fit()
>>> round(float(res.params["emp_growth"]), 1)  # ~1.5
1.5

rotemberg_weights property

rotemberg_weights: DataFrame

Rotemberg weight decomposition by industry.

fit

fit() -> EconometricResults

Fit Bartik IV via 2SLS.

shift_share_se

shift_share_se(iv_result: EconometricResults, shares: ndarray, alpha: float = 0.05) -> EconometricResults

Replace the SE of a shift-share regression with the AKM (2019) SE.

Takes the result of :func:sp.bartik (2SLS with the shift-share instrument) or :func:sp.ssaggregate and recomputes the inference on the last coefficient (the endogenous regressor, or the shift-share variable in a reduced form) with the Adao-Kolesar-Morales exposure-robust formula, exactly as R ShiftShareSE::ivreg_ss / reg_ss and Stata ivreg_ss / reg_ss compute it.

Parameters:

Name	Type	Description	Default
iv_result	EconometricResults	Result of sp.bartik or sp.ssaggregate. These record the outcome, endogenous regressor, shift-share instrument and controls the AKM formula needs; results from other estimators do not, and are rejected.	required
shares	array-like of shape (n, K)	Exposure-share matrix (need not sum to one).	required
alpha	float	Level for the AKM / AKM0 confidence intervals.	0.05

Returns:

Type	Description
EconometricResults	Same point estimates; the last SE is the AKM SE and the diagnostics carry the Homoscedastic / EHW / AKM / AKM0 rows. p-values and CIs are normal-based, as in the reference.

Examples:

>>> import numpy as np, pandas as pd
>>> import statspai as sp
>>> rng = np.random.default_rng(0)
>>> n, K = 80, 5
>>> S = pd.DataFrame(rng.dirichlet(np.ones(K), size=n),
...                  columns=[f"ind{k}" for k in range(K)])
>>> g = pd.Series(rng.normal(0, 1, K), index=S.columns)
>>> bartik = S.values @ g.values
>>> emp = 0.7 * bartik + rng.normal(0, 0.5, n)
>>> wage = 1.5 * emp + rng.normal(0, 0.5, n)
>>> df = pd.DataFrame({"wage": wage, "emp": emp})
>>> iv_res = sp.bartik(df, y='wage', endog='emp', shares=S, shocks=g,
...                    leave_one_out=False)
>>> corrected = sp.shift_share_se(iv_res, shares=S.values)
>>> bool("SE (AKM)" in corrected.diagnostics)
True

ssaggregate

ssaggregate(data: DataFrame, y: str, x: str, shares: ndarray, shocks: Union[str, ndarray, Series] = None, shock_data: Optional[DataFrame] = None, controls: Optional[List[str]] = None, cluster: Optional[str] = None, alpha: float = 0.05, weights: Optional[str] = None) -> EconometricResults

Shift-share IV with AKM (2019) inference and the BHJ shock-level view.

Estimates the location-level regression of y on x (plus controls and an intercept). With shocks given, x is instrumented by the shift-share (Bartik) variable B_i = sum_k s_ik g_k (just-identified 2SLS); without shocks, x is taken to be the shift-share variable itself and the model is fitted by OLS (the reduced form / reg_ss case).

The reported SE of x is the Adao-Kolesar-Morales exposure-robust SE, computed exactly as R ShiftShareSE::ivreg_ss / reg_ss (and Stata ivreg_ss / reg_ss): the control-residualised shift-share variable is regressed on the shares to recover the control-adjusted shocks hX_k, and SE = sqrt(sum_k (hX_k * s_k' e)^2) / RX. The diagnostics also carry that package's other rows (Homoscedastic, EHW, region-cluster when cluster is given, and the AKM0 null-imposed CI).

With shocks given, the Borusyak-Hull-Jaravel shock-level aggregation (what Stata / R ssaggregate produce) is returned in result.data_info["shock_data"] -- per shock s_n (exposure weight, sums to one) and the exposure-weighted means of the control-residualised y and x -- together with the shock-level IV coefficient and its HC0 SE (ivreg2 y (x = g) [aw = s_n], robust) in the diagnostics. When the shares do not sum to one, the shock-level coefficient equals the location-level one only if the sum of shares is among the controls; a warning is raised otherwise.

Parameters:

Name	Type	Description	Default
data	DataFrame	Observation-level data (n rows).	required
y	str	Outcome variable name.	required
x	str	Endogenous regressor (IV mode, shocks given) or the shift-share variable itself (OLS mode, shocks omitted).	required
shares	array-like of shape (n, K)	Exposure-share matrix. shares[i, k] is unit i's exposure to shock k; rows need not sum to one.	required
shocks	str or array-like of shape (K,)	Shock vector, or a column name in shock_data.	None
shock_data	DataFrame	Shock-level DataFrame (K rows) when shocks is a column name.	None
controls	list of str	Location-level controls (an intercept is always added).	None
cluster	str	Location-level cluster variable (e.g. state) for the SE (Reg. cluster) diagnostic. The reported SE is AKM.	None
alpha	float	Level of the AKM / AKM0 confidence intervals in the diagnostics.	0.05
weights	str	Column of non-negative location weights (e.g. ADH's timepwt48). The regression becomes weighted 2SLS / OLS and the AKM scores carry the weights, as ShiftShareSE's w; the BHJ shock-level view uses them as l_weights.	None

Returns:

Type	Description
EconometricResults	Coefficients of the location-level regression; the SE of x is AKM, the controls carry HC1 SEs. p-values / CIs are normal-based (as in the reference). Diagnostics: SE (AKM), SE (AKM0), SE (EHW), SE (Homoscedastic), SE (HC1), AKM / AKM0 CIs, first-stage F, and (IV mode) beta (BHJ shock-level) / SE (BHJ shock-level, HC0).

Notes

Convention differences inside the reference itself, reproduced as is: in IV mode the Homoscedastic row uses RSS / n and EHW has no small-sample factor (it equals HC0); in OLS mode the Homoscedastic row uses RSS / (n - p) and EHW multiplies by n / (n - p). SE (HC1) is the HC1 SE of x from the full regression. AKM0 reports the CI half-width over z_{1 - alpha/2} as its SE; if the CI is unbounded, model_info["akm0_ci_type"] says so.

Examples:

>>> import numpy as np, pandas as pd
>>> import statspai as sp
>>> rng = np.random.default_rng(0)
>>> n, K = 80, 5
>>> shares = rng.dirichlet(np.ones(K), size=n)  # exposure shares, rows sum to 1
>>> g = rng.normal(0.0, 1.0, K)                 # industry-level shocks
>>> bartik = shares @ g                          # shift-share instrument
>>> emp = 0.7 * bartik + rng.normal(0, 0.5, n)   # endogenous regressor
>>> wage = 1.5 * emp + rng.normal(0, 0.5, n)     # outcome
>>> df = pd.DataFrame({"wage": wage, "emp": emp})
>>> df_shocks = pd.DataFrame({"industry_growth": g})
>>> result = sp.ssaggregate(
...     data=df,
...     y="wage",
...     x="emp",
...     shares=shares,
...     shocks="industry_growth",
...     shock_data=df_shocks,
... )
>>> bool(result.params.shape[0] >= 1)
True

shift_share_political

shift_share_political(data: DataFrame, *, unit: str, time: str, outcome: str, endog: str, shares: DataFrame, shocks: Series, covariates: Optional[Sequence[str]] = None, leave_one_out: Optional[bool] = None, alpha: float = 0.05) -> ShiftSharePoliticalResult

Long-difference shift-share IV with AKM SE and Rotemberg weights.

Parameters:

Name	Type	Description	Default
data	DataFrame (long format)	Unit × time panel containing outcome and endog. First and last periods per unit are used to form long-differences.	required
unit	str	Column names.	required
time	str	Column names.	required
outcome	str	Column names.	required
endog	str	Column names.	required
shares	DataFrame (unit × industry)	Exposure-share matrix. Row index must equal the unit IDs.	required
shocks	Series (industry → scalar)	National / supra-unit shifter vector. Index must match the columns of shares.	required
covariates	sequence of str	Pre-treatment covariates (measured at the first period per unit) used for the share-balance diagnostic.	None
leave_one_out	Optional[bool]	Forwarded to :func:sp.bartik. With national shocks and no regional shocks there is no leave-one-out instrument: the default None uses the plain Bartik instrument, True raises.	None
alpha	Optional[bool]	Forwarded to :func:sp.bartik. With national shocks and no regional shocks there is no leave-one-out instrument: the default None uses the plain Bartik instrument, True raises.	None

Returns:

Type

Description

ShiftSharePoliticalResult

estimate / se / ci: 2SLS of the long-differenced outcome on the long-differenced endog instrumented by B_i = sum_k s_ik g_k (intercept only), HC1 SE with normal CI. diagnostics['akm_se'], ['akm_ci'], ['akm0_ci'], ['akm_pvalue']: AKM / AKM0 inference for the same IV (shares as the W matrix, intercept as control). rotemberg_top: alpha_k, beta_k of Goldsmith-Pinkham, Sorkin & Swift (weights sum to one and may be negative). share_balance: F-test of each covariate on the shares (restrictions = rank of [1, shares] minus one).

Examples:

>>> import statspai as sp, numpy as np, pandas as pd
>>> rng = np.random.default_rng(0)
>>> units = range(20); times = range(2); inds = [f'I{k}' for k in range(5)]
>>> shares = pd.DataFrame(rng.dirichlet(np.ones(5), size=len(units)),
...                       index=list(units), columns=inds)
>>> shocks = pd.Series(rng.normal(size=5), index=inds)
>>> rows = []
>>> true_tau = 0.4
>>> for i in units:
...     bartik_i = float((shares.loc[i] * shocks).sum())
...     dx = bartik_i + rng.normal(scale=0.1)
...     y_first = 0.0
...     y_last = y_first + true_tau * dx + rng.normal(scale=0.1)
...     rows.append({'unit': i, 'time': 0, 'y': y_first, 'x': 0.0})
...     rows.append({'unit': i, 'time': 1, 'y': y_last, 'x': dx})
>>> df = pd.DataFrame(rows)
>>> out = sp.shift_share_political(
...     df, unit='unit', time='time',
...     outcome='y', endog='x',
...     shares=shares, shocks=shocks,
... )
>>> abs(out.estimate - true_tau) < 0.3
True

shift_share_political_panel

shift_share_political_panel(data: DataFrame, *, unit: str, time: str, outcome: str, endog: str, shares: Any, shocks: Any, covariates: Optional[Sequence[str]] = None, cluster: str = 'unit', alpha: float = 0.05, fe: str = 'two-way') -> ShiftSharePoliticalPanelResult

Multi-period panel shift-share IV with fixed effects.

Pooled 2SLS with unit / time / two-way fixed effects, using the period-specific Bartik instrument

Z_{it} = sum_k s_{ikt} · g_{kt}

The share matrix can be time-invariant (DataFrame indexed by unit) or time-varying (dict[time → DataFrame]); the shock vector can similarly be scalar-in-time (Series) or time-varying (DataFrame indexed by time / dict[time → Series]).

Parameters:

Name	Type	Description	Default
data	DataFrame (long format)		required
unit	str		required
time	str		required
outcome	str		required
endog	str		required
shares	DataFrame or dict[time → DataFrame]		required
shocks	Series, DataFrame(time × industry), or dict[time → Series]		required
covariates	sequence of str	Time-varying controls.	None
cluster	('unit', 'time', 'twoway', 'shock')	SE type. 'unit' / 'time': one-way cluster-robust (CR0, no small-sample factor; fixest with ssc(adj = FALSE, cluster.adj = FALSE)). 'twoway': Cameron-Gelbach-Miller two-way clustering by unit and time (CR0 components). 'shock': AKM shock-level SE (= ShiftShareSE::ivreg_ss with the FE as controls; shock clusters are industries if the shocks are constant over time, industry x period otherwise).	'unit'
alpha	float		0.05
fe	('two-way', 'unit', 'time', 'none')	Fixed-effect structure (exact within transformation, also on unbalanced panels).	'two-way'

Returns:

Type	Description
ShiftSharePoliticalPanelResult

Examples:

>>> import statspai as sp, numpy as np, pandas as pd
>>> rng = np.random.default_rng(0)
>>> units, times, inds = list(range(30)), [0, 1, 2, 3], list("AB")
>>> shares = pd.DataFrame(rng.dirichlet(np.ones(2), size=len(units)),
...                       index=units, columns=inds)
>>> shocks = pd.DataFrame(
...     rng.normal(size=(len(times), 2)), index=times, columns=inds,
... )
>>> rows = []
>>> tau = 0.3
>>> for i in units:
...     for t in times:
...         b = float((shares.loc[i] * shocks.loc[t]).sum())
...         x = b + rng.normal(scale=0.1)
...         y = tau * x + rng.normal(scale=0.1)
...         rows.append({'u': i, 't': t, 'y': y, 'x': x})
>>> df = pd.DataFrame(rows)
>>> out = sp.shift_share_political_panel(
...     df, unit='u', time='t', outcome='y', endog='x',
...     shares=shares, shocks=shocks,
... )
>>> abs(out.estimate - tau) < 0.15
True

bartik

bartik(data: DataFrame, y: str, endog: str, shares: DataFrame, shocks: Series, covariates: Optional[List[str]] = None, leave_one_out: Optional[bool] = None, regional_shocks: Optional[DataFrame] = None, robust: str = 'hc1', alpha: float = 0.05, weights: Optional[str] = None) -> EconometricResults

Estimate using Shift-Share (Bartik) instrumental variables.

Parameters:

Name	Type	Description	Default
data	DataFrame	Cross-sectional data with one row per region/unit.	required
y	str	Outcome variable.	required
endog	str	Endogenous regressor (e.g., local employment growth).	required
shares	DataFrame or array - like	Share matrix (n_units x n_industries). Rows = regions, cols = industries. Row index must align with data index. A bare 2-D array is accepted; its columns are named k0..k{K-1}.	required
shocks	Series or array - like	National shock vector (n_industries,). Index = industry names. A bare 1-D array is matched to the share columns by position.	required
covariates	list of str	Exogenous control variables.	None
leave_one_out	bool	Compute leave-one-out shocks (exclude own region from national average), which needs regional_shocks -- without the per-region industry growth panel g_k excluding region i cannot be reconstructed. None (default) uses leave-one-out exactly when regional_shocks is supplied; True without regional_shocks raises; False always uses the simple instrument. (Until 1.32 the default was True, which on the basic API only ever warned and fell back.)	None
regional_shocks	DataFrame	Regional industry growth matrix (n_units x n_industries). Row i, column k is the realised growth of industry k in region i. When provided with leave_one_out=True, the instrument uses g_k^{-i} = (sum_j g_{jk} - g_{ik}) / (n - 1) (Borusyak- Hull-Jaravel 2022-style exact leave-one-out). Row index must align with shares; columns must be a superset of shocks.index.	None
weights	str	Column of non-negative analytic weights (Stata [aw=]): weighted 2SLS, HC1 as ivregress ... [aw=w], vce(robust) small, weighted Rotemberg weights; the weights are recorded for :func:sp.shift_share_se, which then computes the weighted AKM SE (ShiftShareSE's w).	None
robust	(hc1, nonrobust)	Standard error type: 'hc1' is the 2SLS sandwich with the n / (n - k) factor (Stata ivregress 2sls, vce(robust) small, R sandwich::vcovHC(type = "HC1")); 'nonrobust' uses sigma^2 = RSS / (n - k) (ivregress 2sls, small). Other values raise. For shift-share (AKM) inference pass the result to :func:sp.shift_share_se.	'hc1'
alpha	float	Significance level.	0.05

Returns:

Type	Description
EconometricResults	2SLS results with Bartik IV diagnostics. The Rotemberg weight table (columns industry, weight = alpha_k, shock, beta = the just-identified IV using industry k's share alone, as Stata bartik_weight reports) is in result.model_info["rotemberg_weights"].

Examples:

>>> import statspai as sp
>>> import numpy as np, pandas as pd
>>> rng = np.random.default_rng(0)
>>> n, K = 80, 4
>>> # shares: DataFrame (regions x industries), shocks: Series (industries)
>>> shares = pd.DataFrame(
...     rng.dirichlet(np.ones(K), size=n),
...     columns=[f"ind{k}" for k in range(K)],
... )
>>> shocks = pd.Series(rng.normal(0.05, 0.02, K), index=shares.columns)
>>> emp = shares.values @ shocks.values + rng.normal(0, 0.01, n)
>>> wage = 1.0 + 2.0 * emp + rng.normal(0, 0.02, n)
>>> df = pd.DataFrame({"wage_growth": wage, "emp_growth": emp})
>>> result = sp.bartik(df, y='wage_growth', endog='emp_growth',
...                    shares=shares, shocks=shocks, leave_one_out=False)
>>> _ = result.summary()