This methods library is the practice layer behind Too Early To Say. It organizes how a real policy question becomes a defensible number through framing, data checks, identification, implementation, verification, interpretation, and evidence preservation.
AI is an instrument inside that practice. Agents handle search, first-pass implementation, reruns, and reconciliation. The economist retains the decisions that determine what the estimate means and whether the claim applies beyond the study setting.
The stages organize the current library by the research decision at stake. Each entry is one worked instance within the complete sequence.
The health-economics contribution study classifies 2,493 published abstracts and shows why the research question must be explicit before implementation begins.
Agent: structures candidate questions. Researcher: selects the estimand and decision use.
The grocery API article reports 6,613 API records. Its run log, cost, retry counts, and zero-loss claim remain under reconciliation.
Agent: collects, reconciles, and flags. Researcher: defines source authority and acceptable loss.
The staggered-adoption case plants an effect of 1.60 and shows why comparison structure matters.
Agent: implements candidate estimators. Researcher: defends identification and the target population.
The methods-to-code article reports a 45-minute transit implementation. No public run record reproduces that timing.
Agent: drafts and refactors code. Researcher: specifies inputs, invariants, and acceptance tests.
The double-machine-learning case recovers 0.965, close to the planted truth.
Agent: runs assertions and known-truth cases. Researcher: approves truths, tolerances, and release claims.
The Medicaid landscape begins with 238 million billing rows and still cannot supply a valid fraud label.
Agent: executes sensitivity checks. Researcher: names what the checks cannot establish.
The school-funding article reports an 8.7% fall after temporary support expires. Public-material status: article only.
Agent: traces supporting evidence. Researcher: states the decision relevance and caveat.
The codebase article reports consolidating a 47-script project. No public inventory, dependency map, or hash manifest reproduces the result.
Agent: assembles and checks the package. Researcher: approves provenance, citation, and handoff.
The five-step guide applies the larger practice to an economist's first project with a coding agent. It stays a focused track inside the methods library.
The complete archive contains 71 methodology articles. Selected studies include public code and documentation; availability varies by study.
Articles that examine a research decision from setup to failure check. Selected studies link to public code and documentation.
A propensity-score match can pass every balance check and still return the wrong number. Observed-covariate balance does not establish that a comparable control existed.
A zero-error pre-treatment fit returns a clean gap of 6.1 against a planted effect of 6.0, and the same zero-error fit returns a wrong gap of 4.3 when no valid counterfactual exists. Pre-treatment fit quality determines whether the post-treatment gap is interpretable.
A global polynomial fit returns a clean, plausible 1.8 where the effect planted at the cutoff is 0.75. A local fit recovers about 0.75. The article explains regression discontinuity in Python and a confounder that the local fit does not adjust for.
Double machine learning in Python: why a naive plug-in estimates 0.55 when the true effect is 1.0, how cross-fitting recovers 0.97, and the confounder it still cannot detect.
When difference-in-differences fits, which assumptions matter, and how the design assumptions fail, shown across worked cases with study-level material status. A decision table maps each situation to the estimator and the corresponding diagnostic requirement.
The Mroz walkthrough reports OLS at 0.1075 and IV at 0.0614. A first-stage F of 55.4 supports relevance only; it cannot establish the independence or exclusion arguments required for a causal interpretation.
An instructional guide to transformation choice and small-sample inference. The simulation estimates and Python rebuild are article-reported and not publicly reproduced.
A conceptual division of labor for AI-assisted econometrics. The exact outputs in its worked examples are article-reported and not publicly reproduced.
Rebuilding an estimator from a paper or package, the code can run without error and still be wrong. A routine that detects the mismatch: name the low-visibility choices in a spec, plant a known truth in a simulation, and read the code against its source.
The article reports a local model comparison. Its scripts, data provenance, saved outputs, and run record are not public, so the exact rankings and metrics are not independently reproduced.
The article reports a local estimator experiment. Its scripts, construction corpus, saved outputs, and run record are not public, so the numerical comparison is not independently reproduced.
The article teaches the estimator through a local HINTS imputation example and a separate simulation. Neither has a public script, data-provenance record, saved output, or run record.
The article classifies 2,493 published health-economics abstracts by contribution scope. Identification and Reframing labels are associated with fitted 1.91x and 2.26x differences on the citations-plus-one geometric scale relative to Calibration; the selected sample does not identify editorial importance or a causal effect.
An iterative-critique workflow built around Medicaid branding estimates and elapsed-time comparisons that are not publicly reproduced.
A repeatable audit for skills, hooks, and memory entries created for older models before they shape current analysis.
A three-layer architecture, rule, gate, and verification, for constraining reasoning-agent behavior when system prompts alone are not enough.
Combining Google Scholar and Semantic Scholar sources with an AI assistant that integrates citation networks and detects reference errors.
A teaching workflow for documenting project conventions in CLAUDE.md. Its session times, prevented-error counts, and savings estimates are not independently timed.
A precise methods paragraph serves as an implementation specification. The transit example and timing comparison are article-reported and not publicly reproduced.
A teaching workflow for mapping dependencies and reorganizing code. The script counts, changed imports, hashes, and timing claims are not publicly reproduced.
The collection safeguards are documented. The article-reported record total, cost, retry log, and zero-loss claim remain under reconciliation.
The article reports an iterative labeling result. Its full-population classification remains under reconciliation with the related policy pages.
Cross-validating SNAP retailer data against multiple authoritative sources documents provenance for each label.
Step-by-step guide to r5py, GTFS data, and multimodal accessibility analysis. The project-specific pair count is under reconciliation.
A conceptual division of labor for AI-assisted econometrics. The exact outputs in its worked examples are article-reported and not publicly reproduced.
Rebuilding an estimator from a paper or package, the code can run without error and still be wrong. A routine that detects the mismatch: name the low-visibility choices in a spec, plant a known truth in a simulation, and read the code against its source.
An instructional guide to transformation choice and small-sample inference. The simulation estimates and Python rebuild are article-reported and not publicly reproduced.
The article reports a local model comparison. Its scripts, data provenance, saved outputs, and run record are not public, so the exact rankings and metrics are not independently reproduced.
The article reports a local estimator experiment. Its scripts, construction corpus, saved outputs, and run record are not public, so the numerical comparison is not independently reproduced.
The article teaches the estimator through a local HINTS imputation example and a separate simulation. Neither has a public script, data-provenance record, saved output, or run record.
The article classifies 2,493 published health-economics abstracts by contribution scope. Identification and Reframing labels are associated with fitted 1.91x and 2.26x differences on the citations-plus-one geometric scale relative to Calibration; the selected sample does not identify editorial importance or a causal effect.
An iterative-critique workflow built around Medicaid branding estimates and elapsed-time comparisons that are not publicly reproduced.
A proposed quarterly audit for skills, hooks, and memory entries created for older models, with hypothetical failure paths to turn into regression tests.
Usage telemetry measures AI adoption; slow public data measures household conditions. The same AI tooling can process both at the cadence each source permits.
A three-layer architecture for constraining reasoning-agent behavior: rule, gate, and verification. The proposed design combines architecture with explicit instructions.
Google Scholar alerts and Semantic Scholar monitor academic literature. An AI coding assistant can combine sources, follow citation networks, and detect reference errors.
Browse all 71 methodology articles by category below.
Official HHS metadata confirms the row count. The article explains the file's screening uses and why its administrative fields do not supply a valid fraud label.
Exclusion lists are often used as fraud-label proxies, but their administrative definition differs from adjudicated fraud.
A guide to billing features, label contamination, and proxy bias. The provider comparisons remain article-reported and under reconciliation.
A guide to temporal validation and interpretable baselines. The model sample, base rate, and classifier metrics remain under reconciliation.
The article classifies 2,493 published health-economics abstracts by contribution scope. Identification and Reframing labels are associated with fitted 1.91x and 2.26x differences on the citations-plus-one geometric scale relative to Calibration; the selected sample does not identify editorial importance or a causal effect.
An iterative-critique workflow built around Medicaid branding estimates and elapsed-time comparisons that are not publicly reproduced.
A teaching workflow for documenting project conventions in CLAUDE.md. Its session times, prevented-error counts, and savings estimates are not independently timed.
A precise methods paragraph serves as an implementation specification. The transit example and timing comparison are article-reported and not publicly reproduced.
A teaching workflow for mapping dependencies and reorganizing code. The script counts, changed imports, hashes, and timing claims are not publicly reproduced.
The collection safeguards are documented. The article-reported record total, cost, retry log, and zero-loss claim remain under reconciliation.
The article reports an iterative labeling result. Its full-population classification remains under reconciliation with the related policy pages.
Cross-validating SNAP retailer data against multiple authoritative sources.
Step-by-step guide to r5py, GTFS data, and multimodal accessibility analysis. The project-specific pair count is under reconciliation.
A regression-residual lesson built around article-reported coefficients and ranks that are not publicly reproduced.
The rest of the library: the full set of AI-assisted research workflows, code tutorials, and method walkthroughs.
Claude Code guides for academic research: CLAUDE.md context files, skills, hooks, MCP servers, context window management, and verification practices. From first session to personal AI infrastructure.
How CLAUDE.md, skills, hooks, and MCP servers combine into a personal research system. Part 5 of the Advanced Tier series.
A 14-article guide to Claude Code: context management, session workflows, agent spawning, and personal AI infrastructure. Estimated 2-3 hours total reading time.
A practical walkthrough of what Claude Code can and cannot do, with prompting patterns and a complete first-task example.
A starter kit for economists using Claude Code: one CLAUDE.md template, a verification checklist, three starter skills, and a week-one glossary.
How CLAUDE.md supplies project context at the start of an AI session.
Understanding how AI context windows work, why sessions reset, and how to work with this fundamental limitation of large language models.
Treating tokens as a finite resource, and knowing when to spawn agents versus work directly.
How Claude Code explores research projects using three core tools: Read (look at a file), Glob (find files by pattern), and Grep (search inside files).
Exploration, implementation, and documentation require different AI prompting strategies. Match the prompt to the phase.
Skills store reusable task instructions that Claude Code can apply when the task matches their trigger.
Hooks are automatic triggers that run without asking - like auto-save, but for research tasks. A power-user feature, entirely optional.
How to connect Claude Code to external data sources like FRED, Census, and Google Scholar for querying those sources through Claude Code.
When to create separate Claude Code helpers for focused work, how to design tasks that are easy to hand off, and patterns for running multiple helpers at the same time.
A conceptual account of how task specification and project context can bound agent behavior. No controlled comparison here isolates those factors from model capability.
A hypothesis that task tolerance and specification shape perceived AI capability, plus the controlled comparison needed to separate those factors from the model.
Verification checks detect unsupported claims before later steps reuse them.
A teaching comparison of chat-based and repository-aware workflows. The cycle count and error story are illustrative scenarios, not measured benchmarks.
A capture routine for handing decisions, open questions, and verification state to the next session.
A Claude Code /insights report supports different lessons at beginner and intermediate levels from the same underlying data.
A setup guide for the public TETS skill skeletons. Some workflows require user-supplied inputs and do not ship with a turnkey example run.
Structured prompts make layout and verification constraints explicit. The article's numerical figure examples are hypothetical teaching inputs.
A case study in form-filling workflows that keep humans in the loop. Browser automation handles navigation and data entry while the human retains final approval.
Tag datasets by the questions they can answer, not just what they contain. A question-first system makes existing datasets searchable by research question.
Python tutorials for applied economics: difference-in-differences, spatial joins, SHAP interpretation, GTFS validation, Census API pipelines, and imbalanced classification.
A statsmodels workflow for event study estimation, with the diagnostics that separate credible estimates from noise.
An instructional workflow for class imbalance and rare-class metrics. Dataset counts, model results, and the named implementation are not publicly reproduced.
An instructional SHAP workflow. The model metrics, importance shares, runtime, and named implementation are not publicly reproduced.
A Python workflow for pulling ACS data from the Census API, including the validation checks that prevent bad data from reaching the analysis.
Six instructional validation layers. The named project inputs, failure counts, and implementation are not publicly reproduced.
A Claude Code skill that generates a structured papers.md block from a DOI or PDF, documenting the estimator, identification strategy, and named assumptions, with a misattribution flag when the bibliography credits…
A public code skeleton for measuring package structure. Seed files, tests, validation outputs, and a public panel are not included, so it is not a turnkey run.
A Claude Code skill that builds a navigable citation network around a single method family, recording the seminal references, the current authors, the recent applications, and papers that…
A case study in tracking California health data releases with Claude Code. Detect new data releases without manual checking across HCAI, DHCS, DOF, and other state agencies.
An article-reported proxy classification. The source data, classification tables, code, and matched outputs are not public.
An illustrative workflow from point-to-polygon joins to local cluster diagnostics. The empirical values and named implementation are not publicly reproduced.
The article reports a 22-fold crime range. No matching public script, run record, or saved output currently reproduces the geographic case.
The article reports a 2.3-fold county comparison. No matching public script, run record, or saved output currently reproduces the rankings.
The pilot and statewide counts, timings, classifications, and findings are article-reported. The original repository no longer resolves.
A sensitivity-analysis lesson built around bank-closure estimates and diagnostics that are not publicly reproduced.
An article-reported TWFE-DiD exercise on SNAP BBCE adoption. The panel, estimates, and diagnostics are not publicly reproduced.
AI-assisted research places a coding agent inside a researcher-defined workflow. The researcher supplies the estimand, variable definitions, data constraints, and acceptance checks. The agent assists with search, implementation, reruns, and documentation. The researcher verifies the output before making the reporting decision. The context-file article shows one instructional setup and labels its savings estimates as independently unmeasured.
r5py provides a Python interface to Conveyal's R5 routing engine. Combined with public GTFS feeds, it can build multimodal travel-time matrices without a proprietary software license. Data preparation and computing resources still require a documented workflow. The r5py tutorial explains the method and labels its project-specific pair count under reconciliation.
The grocery-store classifier article reports 94% accuracy from an iterative labeling exercise. Its full-population classification remains under reconciliation with the related policy pages, so the validation result and population result have separate evidence statuses.
Selected studies include public code and documentation. Availability varies by study and source license. The main analysis repository holds the materials currently available.