Instrumental Variables in Python: Strength Is Not Validity

Instrumental variables handle a regressor that is entangled with the error. An unobserved factor moves both the treatment and the outcome, so a plain regression of one on the other picks up the confounding along with the effect. The fix is to find an instrument that moves the treatment, is uncorrelated with the outcome equation's structural error, and affects the outcome only through treatment. Python returns the 2SLS estimate in a few lines, often alongside a large first-stage F. That statistic establishes relevance alone; validity still depends on the independence and exclusion arguments.

To separate the two, we can plant a known effect and ask which estimator recovers it. One unobserved confounder moves both the regressor D and the outcome Y; one instrument Z moves D and, by construction, is unrelated to the confounder and to the outcome's error. The planted causal effect of D on Y is 2.0.

Why OLS reads the wrong number

Regress Y on D directly and we get 2.79 against the true 2.0, about 40% too high, because D and Y share the unobserved confounder. Adding observed covariates cannot fix this: the confounder is, by assumption, unobserved. The bias is structural, not sampling noise, so collecting more data does not help. At sample sizes of 1,000, 4,000, and 16,000 the OLS estimate holds at 2.80, 2.79, and 2.79.

import numpy as np

# columns: Z (instrument), D (endogenous regressor), Y (outcome)
X = np.column_stack([np.ones(len(D)), D])
b_ols = np.linalg.lstsq(X, Y, rcond=None)[0][1]
print(round(b_ols, 3))            # -> 2.81 on one draw; averages 2.79 over 200 (planted 2.0)

The fix: use only the instrument's variation

Two-stage least squares keeps only the part of D that the instrument moves, which is clean of the confounder. For a single instrument the estimate is the ratio of covariances, the covariance of Z with Y over the covariance of Z with D.

def tsls(Z, D, Y):
    zc = Z - Z.mean()
    return (zc @ (Y - Y.mean())) / (zc @ (D - D.mean()))

print(round(tsls(Z, D, Y), 3))    # -> 2.02 on one draw; averages 2.00 over 200

When we run 2SLS we recover 2.00, with a first-stage F of about 2062, a very strong instrument. With a single binary instrument and heterogeneous effects, the Wald estimand has a local-average-treatment-effect interpretation only under four conditions: the instrument is relevant, independent of potential outcomes and treatment types, excluded from the outcome except through treatment, and monotonic in its effect on treatment. Here the planted effect is the same for everyone, so the local and population averages coincide. Two-stage least squares has no tuning knob to walk in the just-identified case, so there is nothing to sweep for stability. The work is to assess the full set of identifying conditions.

Two failures this simulation makes visible

The first varied condition is relevance, which the first stage can diagnose. Make the instrument barely move the regressor and the 2SLS estimate becomes unreliable: the estimate averages 0.99 but with a standard deviation of 9.2, and the first-stage F drops to about 9. A small F is the diagnostic that flags a weak instrument.

The second varied condition is exclusion, which the first-stage F cannot diagnose. Give the instrument a small direct effect on the outcome, a path that does not run through the regressor, and 2SLS is biased to 2.62 against the true 2.0, about 31% too high. The first-stage F stays about 2051. The estimator still returns a well-defined estimate, and with a single instrument there are no spare comparisons to cross-check. This simulation does not vary independence or monotonicity, so it does not validate either condition.

The assumption no F statistic can certify

The exclusion restriction is the claim that the only route from the instrument to the outcome runs through the regressor. It is an argument about the world, not a quantity in the data. A large first-stage F confirms the instrument is relevant. It is silent on whether a second, direct route exists, and with one instrument there is nothing in the sample that could reveal one.

What to prioritize, before the estimate

Five checks matter more than the first-stage F, in order.

1. Defend independence or conditional exogeneity. Explain why the instrument is unrelated to the structural outcome error and the relevant potential outcomes after any stated conditioning set.
2. Argue the exclusion restriction on the substance. Explain why the instrument reaches the outcome only through the regressor. With one instrument no data test settles it.
3. Read the first-stage F as a relevance check, not a validity check. A large F says the instrument moves the regressor; a small F warns that 2SLS is unstable.
4. Defend monotonicity before using a local-effect interpretation. Name the complier population and explain why the instrument does not move treatment in opposite directions for different units.
5. State the OLS bias we are removing and its likely direction, so the IV estimate can be judged against it.

The estimate is the output, read alongside relevance, independence, exclusion, and, for a local-effect interpretation, monotonicity. The finding depends on whether those conditions are credible for the policy setting, not on the first-stage statistic alone.

References

1. Imbens, G. W., & Angrist, J. D. (1994). Identification and estimation of local average treatment effects. Econometrica, 62(2), 467-475.
2. Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444-455.
3. Staiger, D., & Stock, J. H. (1997). Instrumental variables regression with weak instruments. Econometrica, 65(3), 557-586.

Reproduction

The full simulation framework, including the confounded data generation, OLS and just-identified 2SLS, the sample-size sweep, the weak-instrument case, and the exclusion-violation case, is in the pinned public package. The fixed-seed numerical harness has passed a clean-clone release check and reproduces exactly at that commit. The figure scripts are included as site-maintainer workflows, but they are not part of that clean-clone evidence gate.