Stay in the Loop

Methods-to-Code with AI: An Article-Reported Workflow

The article describes how a precise methods specification can guide implementation. Its project-specific inputs, output, and timing comparison are not publicly reproduced.

A precise methods section can serve as an implementation specification. The project example below is article-reported and not publicly reproduced.

Let's consider what makes this possible, and what still requires human judgment.

The Distance Between Description and Code

Traditional research methods sections describe procedures that someone else must translate into working code. The translation requires understanding domain knowledge, technical requirements, and implicit assumptions that the methods paragraph doesn't state explicitly.

Agent-based coding tools change the translation step. A sufficiently specific methods paragraph can guide a proposed implementation, but access to the inputs, execution, and validation still determine whether the code becomes a working analysis.

What "AI-Ready" Methodology Looks Like

We can examine the article-reported methods paragraph from the food-security project to see what information an implementation would require. Its 408 origins, 4,847 destinations, and resulting output are not publicly reproduced.

"We calculated transit times from census tract centroids to the nearest grocery stores using r5py with OpenStreetMap pedestrian networks and GTFS transit schedules from VTA. For each of 408 census tract centroids, we computed travel time to the closest store among 4,847 validated grocery locations. Travel times represent door-to-door duration including walking to transit, waiting, riding, transferring, and walking from the final stop to the destination. We used Tuesday departure times between 9 AM and 11 AM to capture typical mid-morning shopping trips."

This paragraph contains six implementation-critical elements:

  1. Specific tool named: r5py (not "a routing engine")
  2. Data sources identified: OpenStreetMap, VTA GTFS, validated store file
  3. Article-reported parameters: 408 origins, 4,847 destinations, Tuesday 9-11 AM
  4. Processing logic defined: For each origin, find minimum time to any destination
  5. Travel components specified: Walking, waiting, riding, transferring
  6. Article-implied output: A 408-row table of minimum transit times, not publicly reproduced

The paragraph supplies enough detail for an agent to propose the following implementation steps. This is a specification reading, not evidence that the project-specific run was reproduced:

  • Install r5py
  • Load the OSM network and GTFS feed
  • Read census tract centroids and store locations
  • Configure routing for Tuesday 9-11 AM window
  • Loop through the article-reported 408 origins
  • Find minimum time across the article-reported 4,847 destinations for each origin
  • Save results

The methods paragraph functions as a specification document. A run record and saved output would still be needed to verify the implementation.

Contrast: What Vague Methodology Requires

We can compare this to a less specific version:

"We assessed food access by calculating transit-based travel times to grocery stores using standard routing methods and publicly available transit data."

This describes the same analysis, but lacks implementation detail. An agent (or human programmer) must make numerous decisions:

  • Which routing tool? (Google API? OSRM? r5py?)
  • Which transit data? (Which agency? Current or historical?)
  • Travel time from where? (Centroids? Population-weighted centers? Random points?)
  • To which destinations? (All stores? Nearest store? Stores within threshold?)
  • What time of day? (Peak? Off-peak? Average across day?)

Each decision changes results. Without specification, the agent makes reasonable guesses. Reasonable guesses produce code that runs but may not match research intent.

Article-Reported Implementation Example

The article describes the following interaction with Claude Code. No matching public script, transcript, run record, or saved output currently reproduces the sequence.

Human instruction: "Implement the transit time calculation described in the methods section. The validated store file is at data/processed/stores_validated.csv and census tract centroids are in data/census/tract_centroids.csv."

Agent process:

  1. Reads the methods paragraph from the research document
  2. Identifies required inputs: OSM network, GTFS feed, centroids, stores
  3. Checks for these files in the project directory
  4. Finds that OSM and GTFS files don't exist locally
  5. Asks: "I don't see the OSM network or VTA GTFS feed. Should I download them, or do you have them elsewhere?"
  6. (Human responds: "Download them")
  7. Writes script to download OSM Bay Area extract from Geofabrik
  8. Downloads VTA GTFS from their public feed
  9. Writes transit time calculation script matching specifications:
  • Loads r5py
  • Builds transport network from OSM + GTFS
  • Reads the article-reported 408 centroids and 4,847 stores
  • Configures routing for Tuesday 9-11 AM
  • Calculates minimum transit time from each centroid
  • Saves results to data/processed/transit_times.csv
  1. Runs the script
  2. Reports an output of 408 rows
  3. Reports completion

The article reports approximately 45 minutes of elapsed time, most of it attributed to network building. No public run record currently reproduces that timing.

What Traditional Implementation Required

Before agent-based tools, implementing this same procedure meant:

Day 1 (4 hours):

  • Research transit routing options
  • Discover r5py documentation
  • Understand GTFS data structure
  • Figure out how to download OSM extracts
  • Set up Java environment for r5py
  • Debug installation issues

Day 2 (3 hours):

  • Write data loading code
  • Debug coordinate system mismatches
  • Handle GTFS file format variations
  • Write routing loop
  • Debug memory issues in the article-reported 408 x 4,847 example

Day 3 (2 hours):

  • Validate output against manual spot-checks
  • Fix off-by-one errors in minimum calculation
  • Add progress logging
  • Document the code

The article estimates approximately 9 hours for someone familiar with Python, with longer setup for someone learning the tools. That counterfactual was not measured in a public benchmark.

It then reports 45 minutes of mostly automated work plus 10 minutes of human instruction and verification. Those timings are illustrative, article-reported values, not a publicly reproduced comparison.

The Division of Labor

Implementation automation does not eliminate research judgment. Three categories of decisions remain distinctly human:

1. Research Design Decisions

The methods paragraph specifies "Tuesday 9-11 AM" for departure times. This choice reflects a research question: What does mid-morning shopping access look like?

Alternative valid choices:

  • Peak commute hours (6-9 AM, 4-7 PM)
  • Weekend schedules
  • Average across all times
  • Minimum across day (best-case access)

Each choice answers a different question. The agent can implement any of them efficiently. It cannot determine which question matters for the research.

2. Interpretation of Results

The article reports a file with 408 transit times and summary statistics of 23.4 minutes mean, 18.7 minutes median, and 67.2 minutes maximum. No saved public output reproduces the row count or statistics.

What it does not determine: Whether 67 minutes constitutes a meaningful barrier. Whether the 3.6x difference between fastest and slowest access is policy-relevant. Whether these times suggest that improving transit frequency would reduce food insecurity.

These interpretations require domain knowledge about:

  • How long people will travel for groceries
  • Whether transit times compete with car times or walking times
  • What other barriers exist (cost, cultural appropriateness of stores)
  • Whether the spatial pattern suggests actionable interventions

3. Validation Strategy

The article describes an agent producing 408 values. That output is not public, and even a reproduced output would require human guidance to determine whether the values are reasonable.

Validation requires:

  • Spot-checking specific tracts against Google Maps (do the times seem right?)
  • Comparing to prior studies (are these consistent with known patterns?)
  • Checking edge cases (why does tract X show 67 minutes? Is that real or an error?)
  • Verifying that routing parameters match real behavior (do people actually make 2-transfer trips?)

We can ask the agent to perform specific validation tasks ("spot-check the 5 longest transit times against Google Maps"), but designing the validation strategy is human work.

Time Comparison

Article-reported traditional vs. agent-assisted time
10.5h traditional estimate
0.7h agent-assisted report
93% reported reduction

Evidence limit: The timing values in this comparison are article-reported and no public run record currently reproduces them.

Article-reported time comparison by task; not publicly reproduced
Task Traditional (hours) AI-Assisted (hours) Reduction
Tool research and setup 4.0 0.2 95%
Data acquisition 1.0 0.0 100%
Code implementation 3.0 0.2 93%
Debugging and validation 2.0 0.3 85%
Documentation 0.5 0.0 100%
Total 10.5 0.7 93%

In the article-reported comparison, the largest reductions appear in setup, implementation, and documentation. The table is illustrative and not a publicly reproduced benchmark; verification still requires human judgment about what constitutes "correct."

The Broader Principle

The transit time example generalizes. If we write methods sections with implementation-level specificity:

Data collection:

"We collected store locations using Google Places API, querying for 'grocery store' within a 50km radius of each county centroid, retrieving name, address, coordinates, rating, review count, and place types for each result."

Classification:

"We trained an XGBoost classifier on 400 manually labeled locations (200 grocery stores, 200 non-grocery). Features included: Google place types (binary indicators), name substring matches (market, foods, liquor, gas, 7-Eleven), log review count, and rating."

Index construction:

"We constructed vulnerability scores as: 0.3x(poverty rate) + 0.3x(SNAP rate) + 0.2x(vehicle access) + 0.2x(median transit time), min-max normalizing each component to [0,1] before weighting."

Each description supplies concrete implementation choices: API, query parameters, features, algorithm, or weights. That specificity narrows what an agent must infer, but it does not verify that any project-specific pipeline ran correctly.

What This Changes About Research Workflow

When implementation costs fall, research iteration patterns can change.

Traditional workflow:

  1. Design analysis
  2. Implement (days to weeks)
  3. Run analysis
  4. Discover limitation or alternative approach
  5. Decide if re-implementation worth the time cost
  6. Often: stick with initial approach because re-coding is expensive

Agent-assisted workflow:

  1. Design analysis
  2. Implement (minutes to hours)
  3. Run analysis
  4. Discover limitation or alternative approach
  5. Re-implement immediately
  6. Compare approaches
  7. Iterate until satisfied

The key difference: Step 5. When re-implementation is cheap, we can actually test the alternative approaches we think of. Robustness checks stop being theoretical ("we could try X") and become practical ("let's run it both ways").

What Still Requires Human Expertise

Three categories of work remain irreducibly human:

  1. Deciding what to measure: Transit times to nearest grocery store vs. transit times to high-quality grocery store vs. number of stores within 30 minutes. Each captures something different. The agent cannot determine which matters.
  2. Interpreting what results mean: A 3.6x difference in transit times is a number. Whether it represents a meaningful barrier to food access requires understanding of shopping behavior, household constraints, and existing literature.
  3. Designing the research question: The entire food security analysis starts from asking whether geographic access or economic access drives food insecurity more strongly. The agent cannot formulate this question. It can help answer it once asked.

Summary

A well-specified methods section can guide an agent-based implementation. The closer we get to implementation-level detail in research documentation, the fewer choices the agent must infer before code can be run and validated.

This creates a positive feedback loop: Writing precise methods sections makes implementation faster. Fast implementation makes testing alternative approaches practical. Testing alternatives improves research quality. Higher quality research requires clearer documentation.

The limiting factor shifts from "how long will this take to code?" to "what exactly should we measure?" That's the question that always should have dominated research workflow.

For the step-by-step version of this move, see the code generation step of our Start Here guide, which shows how to move from a methods paragraph to testable code with the agent in the loop.

About This Series:

This series explores practical applications of agent-based coding in applied economics research: reducing copy-paste iteration cycles, validating large datasets with minimal training data, building robust API collection pipelines, reorganizing research codebases, integrating AI into writing workflows, and bridging methodology documentation with implementation.

The article argues that these tools can compress implementation time. Its timing comparison is not publicly reproduced. The broader workflow still leaves research judgment, interpretation, design, execution, and validation with the researcher.

AI Disclosure: This article was written with AI assistance using Claude Code. Approximately 35% of text was AI-generated (primarily structure and routine explanations). The project example and time comparison are article-reported and not publicly reproduced. Final editing and voice remain human.

Public materials: No matching public implementation, run record, or saved output currently reproduces the transit example or timing comparison.

Next in series: 7 Copy-Paste Cycles to 1 Command shows what changes when AI can read your entire codebase.

Suggested Citation

Cholette, V. (2025, October 8). Methods-to-code with AI: An article-reported workflow. Too Early To Say. https://tooearlytosay.com/research/methodology/methodology-to-code-ai/
Copy citation