Cleaning a Research Codebase: An Article-Reported Example

After three months of iterative analysis, a research project directory might contain dozens of Python scripts with names like analysis_final_v2_FIXED.py. This teaching example uses an article-reported 47-script project, but no public inventory currently reproduces that count. Without a dependency record, the order in which scripts should execute can be difficult to recover.

Basic questions become unanswerable: Which version produces current results? What order should scripts run? Which data files are inputs versus intermediate outputs?

---

The Starting Point

ARTICLE-REPORTED EXAMPLE DIRECTORY (not publicly reproduced)

food_security/
├── analysis.py
├── analysis_v2.py
├── analysis_final.py
├── analysis_final_FIXED.py
├── census_data.py
├── census_data_new.py
├── clean_stores.py
├── clean_stores_backup.py
├── transit_calc.py
├── transit_calc_WORKING.py
└── ... (37 additional files)

The naming tells a story: analysis.py was the original, analysis_v2.py added features, analysis_final.py was supposed to be done, analysis_final_FIXED.py patched a bug discovered later. But which one actually runs? Which produces the current results file?

---

Dependency Mapping

Claude Code can trace data flow through all scripts. It reads every file, identifies import statements and file I/O operations, and builds a dependency graph:

ARTICLE-REPORTED EXAMPLE (not publicly reproduced)

ACTIVE SCRIPTS (15):
- clean_stores.py → stores_cleaned.csv
- census_data_new.py → census_merged.csv
- transit_calc_WORKING.py → transit_times.csv
- analysis_final_FIXED.py → results.csv

DEPRECATED (32):
- analysis.py: Original version, superseded
- analysis_v2.py: Superseded by analysis_final
- analysis_final.py: Contains bug at line 234 (division by zero when tract has no stores)
- census_data.py: Uses obsolete variable names (renamed in ACS 2022)
- transit_calc.py: References stores_raw.csv, which no longer exists
- clean_stores_backup.py: Candidate duplicate; verify with a recorded hash

The article describes 15 of 47 scripts as necessary for the current analysis and classifies the remainder as obsolete versions, backups, or broken code. That classification is useful as a worked example, but no public dependency inventory currently reproduces it.

---

The Reorganized Structure

food_security/
├── README.md
├── CLAUDE.md
├── data/
│   ├── raw/           # Immutable source data
│   ├── processed/     # Cleaned intermediate files
│   └── output/        # Final results
├── scripts/
│   ├── 01_download_census.py
│   ├── 10_clean_stores.py
│   ├── 20_process_census.py
│   ├── 30_calculate_transit.py
│   ├── 40_calculate_vulnerability.py
│   └── 50_generate_figures.py
├── src/               # Shared utility functions
└── archive/           # Deprecated code (preserved)

Numbered prefixes indicate execution order. The gaps (01, 10, 20...) leave room to insert new scripts without renumbering everything. The data directory structure distinguishes inputs from outputs: raw/ is never modified, processed/ holds intermediate files, output/ contains final results.

---

What Gets Updated

The article reports the following project changes. A public diff or run record is not available to reproduce the counts:

• 47 files reportedly moved to appropriate locations
• 15 active scripts reportedly renamed with numbered prefixes
• 89 import statements reportedly updated to reflect new paths
• Duplicated utility code extracted into src/utils.py
• README.md generated with project overview
• CLAUDE.md generated with data conventions for future agent sessions

---

Designing the Counterfactual Test

Reorganization should change file locations and names without changing analytical results. But how do you verify this? The question is fundamentally counterfactual: "What would the output be if I hadn't reorganized?"

The test design:

1. Before reorganization: Run the full pipeline, save all output files, compute SHA256 hashes
2. Reorganize: Move files, rename scripts, update imports
3. After reorganization: Run the pipeline again, compute SHA256 hashes of outputs
4. Compare: Hashes should match exactly

This works when the reorganization changes only file paths and the pipeline is deterministic. A mismatch signals a difference that needs investigation, but a match is meaningful only when the underlying files, commands, and generated manifest are recorded.

Illustrative manifest format (placeholder values):
Before reorganization:
  results.csv: SHA256 = <recorded-hash-a>
  vulnerability_scores.csv: SHA256 = <recorded-hash-b>
  figures/map_vulnerability.png: SHA256 = <recorded-hash-c>

After reorganization:
  data/output/results.csv: SHA256 = <recorded-hash-a>
  data/output/vulnerability_scores.csv: SHA256 = <recorded-hash-b>
  data/output/figures/map_vulnerability.png: SHA256 = <recorded-hash-c>

In a reproducible release, matching recorded hashes would support the conclusion that the saved outputs did not change. This article does not publish the underlying files or manifest, so it does not establish a verified match.

---

When Hashes Don't Match

Consider an illustrative troubleshooting case in which the vulnerability map hash differs because the figure-generation script embeds changing metadata. The visual content can remain the same while the bytes differ.

One possible fix is to remove non-deterministic metadata during export, then regenerate the manifest and inspect any remaining differences. The exact export option depends on the file format and plotting library. A passing comparison should be reported only with the files, command, environment, and manifest needed to reproduce it.

This is the value of designing explicit counterfactual tests. Without hash comparison, a subtle bug (wrong file path, missing data, changed parameter) might go unnoticed until much later.

---

Time Investment

Article-reported time estimates, not independently timed

Task	Reported estimate
Dependency analysis	20 min
Reorganization planning	15 min
File operations and import updates	30 min
Counterfactual testing	25 min
Documentation generation	15 min
Total	1 hr 45 min

The article reports a total of 1 hour 45 minutes. No timing log is public, so the values should be read as planning estimates rather than a measured benchmark. Actual effort depends on repository size, test coverage, dependency complexity, and the condition of the existing pipeline.