The article reports a vulnerability index of 0.595 for Merced County and 0.260 for San Francisco, a 2.3-fold difference that appears to say something about food access. 1 Those values have not been publicly reproduced. Used here as a teaching case, the gap can conflate two distinct phenomena: food access itself, which policy could potentially influence, and economic structure, including income, density, housing costs, and development patterns.
A composite index can move with both measured access and economic structure. Wealth may be associated with store locations, land use, and the resources households use to cope with poor access. Vehicle ownership is relevant to household exposure, but it is not part of the proximity-based mobility-desert classification used elsewhere on this site. The research question here is narrower: how do county scores compare after conditioning on a stated set of covariates?
We can address this through residualization: statistically controlling for structural factors to isolate variation unexplained by those specific controls. 2 This unexplained variation might reflect policy choices, unmeasured structural factors, geographic constraints not captured by density alone, measurement error, or other factors. This article explains how the approach works, presents article-reported results from 58 California counties, and clarifies what residuals can and cannot establish.
The Measurement Problem
Consider two counties:
County A (wealthy, dense):
- Median income: $130,000
- Car ownership: 92%
- Population density: 5,000/sq mi
- Illustrative county mean vulnerability index: 0.20
County B (moderate income, suburban):
- Median income: $65,000
- Car ownership: 85%
- Population density: 800/sq mi
- Illustrative county mean vulnerability index: 0.45
County A has a lower composite score, but the difference may reflect wealth, density, the store and transit network, or other conditions. County B's higher score may likewise combine economic hardship with a more difficult geography for fixed-route service. A raw cross-county comparison does not separate those pathways.
The mobility-desert classification used elsewhere on this site is based on proximity to stores and transit, not on vehicle ownership. Owning a car may change a household's ability to cope with poor access, but it does not mechanically change a tract's mobility-desert classification. In the regression described here, car ownership is a separate explanatory covariate for a composite county vulnerability index.
The Confounding Problem in Food Access
This is an adjustment problem, not by itself a causal design. 3 Wealth, car ownership, land use, store locations, and transit service can be related, but the measurement pathways need to remain distinct:
How wealth shapes what we measure:
Wealth → Car Ownership → Ability to cope with poor access (household exposure)
Meanwhile...
Land use + store locations + transit network → Measured access components → Composite vulnerability
The first pathway can describe exposure or coping capacity. The second supplies the proximity measures used in the access classification. Residualization can condition on measured covariates, but it cannot identify a policy effect or determine which pathway caused a county's score.
The same dynamic affects traditional food desert metrics. Wealthy areas have higher retail density (more grocery stores per capita) partly because the market supports more stores. The low food desert rate might reflect market conditions rather than policy success.
The Residualization Approach
Here's what the model changes: instead of comparing raw county scores alone, it compares scores conditional on a specified set of measured economic covariates.
To construct that descriptive comparison, we use residualization. The result depends on the controls and functional form selected for the model.
Step 1: We predict what each county's vulnerability should be based on its economic and demographic profile alone.
Step 2: We compare what we actually observe to what we predicted.
Step 3: The difference is the county's residual under that specification. It records whether the observed score is above or below the fitted value; it does not say why.
Implementation
We estimate a regression model:
Vulnerability Index = Baseline + (Income effect) + (Density effect) + (Car Ownership effect) + (Housing Burden effect) + Unexplained
Or in statistical notation:
Vulnerability = β₀ + β₁(Income_per_$10K) + β₂(Density_per_1000) + β₃(Car_Ownership) + β₄(Housing_Burden) + ε
Where:
- Income is measured in $10,000 units; density is measured in thousands of persons per square mile; car ownership and housing burden are percentage-point variables
- ε (epsilon) is the residual, capturing unexplained variation
For each county:
- Predicted value: What we'd expect given its structural factors
- Actual value: What we observe
- Residual: Actual minus predicted
What We Learn From the Results
Negative residual: Observed vulnerability is below the value fitted from income, density, car ownership, and housing burden. The residual may reflect omitted context, measurement error, or model misspecification; this regression does not identify the cause.
Positive residual: Observed vulnerability is above the value fitted from income, density, car ownership, and housing burden. The residual may reflect omitted context, measurement error, or model misspecification; this regression does not identify the cause.
Near-zero residual: Observed vulnerability is close to the fitted value under this specification. That does not establish that the included covariates explain the underlying mechanism.
Results: How Rankings Change
The article reports several ranking shifts after applying this approach to 58 California counties. The tables are retained to teach interpretation, but the values have not been publicly reproduced and should be treated as provisional.
Counties with Highest Vulnerability (Raw Scores)
| Rank | County | Raw Index | Residual | Adjusted Rank |
|---|---|---|---|---|
| 1 | Merced | 0.595 | +0.08 | 3 |
| 2 | Alpine | 0.531 | +0.12 | 1 |
| 3 | Tulare | 0.489 | +0.04 | 8 |
| 4 | Kern | 0.476 | +0.02 | 12 |
| 5 | Imperial | 0.471 | +0.06 | 5 |
| 6 | Madera | 0.458 | +0.03 | 10 |
| 7 | Kings | 0.455 | +0.05 | 6 |
| 8 | Fresno | 0.447 | -0.01 | 18 |
| 9 | Colusa | 0.442 | +0.09 | 2 |
| 10 | Glenn | 0.438 | +0.07 | 4 |
Counties with Lowest Vulnerability (Raw Scores)
| Rank | County | Raw Index | Residual | Adjusted Rank |
|---|---|---|---|---|
| 49 | Marin | 0.275 | +0.02 | 48 |
| 50 | Santa Clara | 0.273 | +0.03 | 45 |
| 51 | Contra Costa | 0.272 | -0.01 | 52 |
| 52 | Orange | 0.271 | +0.01 | 50 |
| 53 | Santa Cruz | 0.269 | -0.02 | 54 |
| 54 | Ventura | 0.268 | +0.00 | 51 |
| 55 | San Mateo | 0.268 | -0.02 | 55 |
| 56 | Placer | 0.264 | -0.03 | 56 |
| 57 | El Dorado | 0.261 | -0.04 | 57 |
| 58 | San Francisco | 0.260 | -0.05 | 58 |
Article-Reported Findings
The findings in this section have not been publicly reproduced.
Finding 1: Most Reported Rankings Change Little
In the article-reported output, raw and residualized rankings are similar for most counties. The article interprets this as structural factors and measured outcomes moving together.
The article reports a 0.89 correlation between raw and residual rank. Treat that value as provisional until a public package can reproduce it.
Finding 2: Some Reported Rankings Move
The article reports a few counties where the descriptive story changes after adjustment:
Fresno drops from 8th to 18th highest vulnerability (improves 10 ranks):
- Raw index: 0.447 (high vulnerability)
- Residual: -0.01 (slightly lower than expected)
- Article-reported interpretation: Fresno's measured vulnerability is slightly lower than this model predicts from its included structural variables.
Santa Clara rises from 50th to 45th (moves up 5 ranks in vulnerability):
- Raw index: 0.273 (low vulnerability)
- Residual: +0.03 (slightly higher than expected)
- Article-reported interpretation: Santa Clara's measured vulnerability is slightly higher than this model predicts from its included structural variables.
Finding 3: San Francisco Is Reported Below Its Predicted Value
The article places San Francisco 58th in both raw and residualized rankings and reports a -0.05 residual. In this model, that means the measured outcome is below the prediction from included controls; it does not identify why.
The article notes 65% car ownership and offers transit, walkability, food-access policy, and omitted geography as possible explanations. Those are hypotheses, not findings established by the model.
Finding 4: Alpine Is Reported Above Its Predicted Value
The article places Alpine County 2nd in raw vulnerability (0.531) and 1st after residualization (+0.12 residual). These values are not publicly reproduced.
The article reports only 2 census tracts and roughly 1,200 residents for Alpine, so even a reproduced residual would warrant caution. A large residual in such a small unit could reflect genuine barriers or noise; it identifies a case to investigate, not a definitive finding.
What We Can (and Can't) Learn From Residuals
Residuals tell us how much a county's observed outcome differs from the value fitted from income, density, car ownership, and housing burden. They do not tell us why.
We might be seeing:
-
Policy-related factors: Transit investment, land use decisions, food assistance coordination. But we can't prove this from residuals alone.
-
Unmeasured structural factors: Hills and waterways that make transit harder, street networks that help or hurt walkability, historical development patterns.
-
Measurement limitations: Small samples in rural counties, timing mismatches between data sources, census tract boundaries that don't match real neighborhoods.
The key limitation: A large positive residual means "vulnerability is higher than our model predicts." It does NOT mean "policy failed." To make that kind of causal claim, we'd need different research approaches: tracking changes over time, comparing similar counties with different policies, or detailed case studies.
What residuals do accomplish: They provide a different descriptive comparison, conditional on the included covariates. Instead of asking only which county has the lowest raw score, we can ask which observed scores sit above or below this model's fitted values. That can identify cases for follow-up, not establish what is working.
Model Specification Details
Variables
Dependent variable:
- County mean vulnerability index (0-1 scale)
Structural controls:
- Median household income (continuous, $10,000s)
- Population density (continuous, thousands of persons per square mile)
- Car ownership rate (continuous, percentage)
- Housing cost burden (continuous, % income spent on housing)
Estimation
Evidence note: the fit statistics in this table are article-reported and not publicly reproduced.
| Specification | R² | N |
|---|---|---|
| Income only | 0.51 | 58 |
| Income + Density | 0.67 | 58 |
| Full model | 0.81 | 58 |
The article reports that the full model explains 81% of between-county variance, leaving 19% in the residual. Those percentages are provisional pending reproduction.
Coefficients
Evidence note: coefficients, standard errors, and p-values are article-reported and not publicly reproduced.
| Variable | Coefficient | Robust SE | p-value |
|---|---|---|---|
| Intercept | 0.62 | 0.09 | <0.001 |
| Income (per $10K) | -0.024 | 0.005 | <0.001 |
| Density (per 1000) | -0.008 | 0.003 | <0.01 |
| Car Ownership (%) | -0.003 | 0.001 | <0.05 |
| Housing Burden (%) | +0.002 | 0.001 | 0.08 |
In the article-reported model, higher income, density, and car ownership have negative coefficients, while housing burden has a positive coefficient with a reported p-value of 0.08. These associations are not publicly reproduced.
A note on the article-stated uncertainty calculation: The article reports robust standard errors rather than conventional ones. 4 HC1 allows the residual variance to differ across counties and applies a degrees-of-freedom correction. It does not weight counties by population, correct measurement error, or make county means equally precise. If county means have different sampling precision, that requires justified weights or a model that represents the measurement process.
Limitations
Small Sample Size
The article reports wide residual confidence intervals in a 58-county model. Even if reproduced, that sample size would argue against treating the rankings as definitive.
Model Specification Sensitivity
The article reports the following specification checks; none is publicly reproduced:
-
Using poverty rate instead of income: The article reports a 0.94 correlation with the main specification. That provisional result suggests stability to this one modeling choice, not robustness to all reasonable choices.
-
Adding land area as a control: The article reports negligible improvement in explanatory power. That claim is not publicly reproduced.
-
Excluding housing burden: The article reports minimal changes to the results. That claim is not publicly reproduced and does not establish that housing burden is unimportant.
The article characterizes these checks as reasonably stable but not identical across specifications. A reproducible package is needed before treating that robustness claim as verified.
Cross-Sectional Limitation
Comparing counties at a single point in time means we can't really say much about:
- Whether counties with good residuals achieved them through policy or started with advantages we didn't measure
- Whether both residuals and policy reflect some third factor we're not seeing
- Whether residuals are stable over time or fluctuate year to year
Tracking these patterns over multiple years would give us much stronger evidence.
Ecological Fallacy
When we analyze data at the county level, we learn about counties as units. But counties contain multitudes. A county with a favorable residual might still have neighborhoods facing severe food access barriers—our analysis can't see that variation. And we definitely can't draw conclusions about individual residents from county-level patterns. What's true on average for a county may not be true for the people living there.
Research Applications
What we've done here can inform research in several ways:
1. Case Study Selection
Counties with extreme residuals are natural candidates for deeper investigation:
- Large negative residuals: What's going well here? Is it policy, geography, or something we haven't measured? Case studies can dig into the specifics.
- Large positive residuals: What barriers exist beyond economic disadvantage? Geographic isolation? Infrastructure gaps? Historical underinvestment?
- Near-zero residuals: Less to explain—outcomes match predictions.
2. Fairer Comparisons
Instead of comparing all counties head-to-head, we can group counties by measured economic profile and compare within groups. This holds the included factors more nearly constant and shows which scores are above or below peers under that grouping. Counties may still differ in unmeasured ways, so the comparison remains descriptive.
3. Change Tracking
Tracking residuals over time reveals whether counties improve or decline relative to expectations:
- Declining residual → improving relative to structure
- Rising residual → facing new challenges relative to structure
- Stable residual → consistent performance
4. Equity Analysis
We can calculate residuals for demographic subgroups within counties:
- Majority-minority tracts vs. majority-white tracts
- High-poverty vs. low-poverty tracts
- This reveals whether structural controls mask disparities
Interpreting County Residuals
The article-reported residuals illustrate how adjusted metrics can change the descriptive reading of county outcomes. The specific county values below remain provisional:
Low raw vulnerability ≠ Policy success. The article reports 0.268 for San Mateo and a near-zero residual. In the stated model, that means the outcome is close to the fitted value; it does not establish policy performance.
High raw vulnerability ≠ Policy failure. The article reports 0.447 for Fresno and a small negative residual. In the stated model, that means the outcome is slightly below the fitted value; it does not establish policy performance.
Large positive residuals warrant investigation. Alpine and Colusa show higher vulnerability than their economic profiles predict. This could reflect real barriers—geographic, infrastructural, or policy-related—or it could reflect measurement challenges in small rural counties. Worth investigating, but not proof of failure.
Large negative residuals identify cases to investigate. The article reports lower-than-predicted values for San Francisco and El Dorado. A residual alone cannot establish that a policy or other beneficial mechanism caused the difference.
Conclusion
Residualized metrics provide additional information by adjusting for measured structural differences. They allow comparisons between counties with different baseline characteristics.
For the article-reported California food-access case, which has not been publicly reproduced:
- The model reports 81% of county-level variation associated with income, density, car ownership, and housing burden
- The model reports 19% remaining in the residual after including those factors
- This unexplained variation might reflect policy choices, unmeasured structural factors, geographic constraints, measurement error, or combinations of these factors
- The article reports that most rankings remain similar after residualization
- The article reports that a few counties move substantially relative to the fitted values
- Determining why these counties show different-than-expected patterns requires research designs beyond cross-sectional regression
We can apply this approach to many policy domains where outcomes correlate with structural factors. Residualization adjusts for measured structural differences, improving comparability. However, it cannot distinguish whether unexplained variation reflects policy choices, unmeasured context, or other factors. That determination requires additional research methods: longitudinal analysis, case studies, quasi-experimental designs, or direct measurement of policy implementation.
Bottom line: Residualized comparisons ask whether county scores differ from values fitted from income, density, car ownership, and housing burden. The answer is descriptive. The causal question, why they differ and what would change those differences, requires different analytical approaches.
Public Materials
Article only. The related public folder contains documentation and placeholders, but no matching analysis script, frozen input manifest, run record, or saved output that derives the reported model, coefficients, residuals, rankings, or robustness checks.
Notes
[1] The article states that the county index was calculated in November 2025 using five weighted components. No matching public script, frozen inputs, or saved output derives the reported values. ↩
[2] Residualization approach follows methods in Angrist & Pischke (2008), Mostly Harmless Econometrics. Applied to food access measurement. ↩
[3] Pearl, J. (2009). Causality: Models, Reasoning, and Inference (2nd ed.). Cambridge University Press. ↩
[4] White, H. (1980). "A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity." Econometrica, 48(4), 817-838. https://doi.org/10.2307/1912934. The HC1 estimator applies a degrees-of-freedom correction to the original HC0 estimator. ↩
Tags: #FoodSecurity #Econometrics #ResidualizedIndex #CausalInference #Methodology #PolicyAnalysis #California
Next in this series: Scaling up, covering the computational and methodological challenges of expanding from 7 counties to statewide analysis.
Suggested Citation
Cholette, V. (2025, November 30). Residualized accessibility index: An article-reported case. Too Early To Say. https://tooearlytosay.com/research/methodology/residualized-accessibility-index/Copy citation