Engineering Deep Dive

An Assembly Line for Legacy Code

How six phases of scoped AI agent sessions, coordinated through filesystem locks, an append-only knowledge base, adversarial validation, and human gates, turn a legacy storefront into validated evidence, buildable studies, and finally verified code.

Updated September 2026 ยท About a 7 minute read

Private employer Details are sanitized; architecture and outcomes are described without proprietary identifiers.

TL;DR

  • The only complete spec of the legacy storefront is its code; agents can read all of it but overclaim, so the pipeline wraps them in locks, adversarial review, and human gates.
  • Six phases, each session owning one feature in one phase, coordinated through the filesystem and an append-only knowledge base rather than a conversation.
  • 108 features extracted and reviewed, 39 design studies drafted in three days, and execution shipping feature by feature: the unit of AI work is a claim small enough to verify.

Context

The company is rebuilding its legacy PHP storefront on a modern platform, and the only complete specification of what the storefront does is the storefront itself: years of accumulated routes, includes, business rules, and quiet workarounds that no document describes and no one person remembers. Rebuilding without reading all of it means shipping a new system that silently drops behavior the business depends on. Reading all of it the traditional way is an archaeology project measured in quarters of senior engineering time.

AI agents change the economics of the reading, but not in the naive way. An agent session will happily read code all day, and it will also happily overclaim: assert a variable is dead when its reader lives in a file outside the scope it examined, or call a route unclaimed because prose said so when the filesystem said otherwise. Treating agent output as truth is how an AI-assisted migration becomes a confidently wrong one.

So the system under design here is not "AI reads the code." It is a pipeline that treats agents as fast, tireless, fallible readers, and wraps them in the verification machinery their fallibility demands: locks, adversarial review, ground-truth checks, and human gates.

Constraints

  • Sessions are parallel and isolated. Many agent sessions run concurrently, including scheduled ones, and none can see another's conversation. Coordination has to live somewhere they all share, which means the filesystem, not a chat.
  • Every claim must be checkable. An assertion about legacy behavior is only useful if a later session, or a human, can verify it against the code, the database, or git history. Findings must cite paths, and absence of evidence gets recorded as a finding of its own.
  • Human attention is the scarce resource. The point of the pipeline is that engineers stop reading legacy code. Whatever reaches a person has to arrive as a decision to make, not a pile to wade through.
  • The legacy system is production. Agents read it, measure it through a read-only database connection, and dig through git history, but never modify it. Verification had to be built entirely from non-destructive tools.

Options considered

Option A: the big rewrite prompt. Hand the repository to a large model and ask for the new system. This is the approach the tooling seems to invite, and it fails on provenance: the output is an unreviewable everything-diff with no trail from any behavior back to the legacy code that justified it. When the new system disagrees with the old one, there is no way to know whether the model fixed a bug or invented one.

Option B: human-led analysis with AI assistance. Engineers read the legacy code and agents help summarize. Correctness is fine; throughput is not. Hundreds of entangled features times days of senior attention each is the exact archaeology project the business could not afford, and in practice the rebuild would end up scoped by whichever features lived loudest in institutional memory.

Option C: an assembly line of small, verified claims. The chosen design inverts the big prompt. No agent ever holds the whole problem. Each session owns one feature in one phase, produces a small artifact with citations, and hands it forward. Verification is not a final review but a phase of its own, staffed by agents whose explicit brief is to assume the previous agent was wrong. Humans referee disputes and answer questions; they do not read the monolith.

The design

The pipeline runs in six phases, each producing artifacts the next phase consumes without conversation. Extraction agents work in the legacy repository, claiming one feature each by atomic folder creation, the filesystem itself acting as the lock, and documenting what the code actually does as rule-level findings with citations. Review agents then read each extraction against the new platform and file a verdict, keep, partial, or drop, citing both codebases. Validation agents take a second, adversarial pass: prove or disprove each claim, correct reports in place, and escalate disagreements as formal verdict challenges rather than silent edits.

Two mechanisms hold the fleet together. The first is an append-only knowledge base: every session must read the onboarding document before touching anything, and closes by writing its corrections and lessons back, superseding earlier claims explicitly rather than deleting them. The lessons are blunt and earned, on the order of "a published-but-unrun command is a claim, not a result" and "prose saying unclaimed is resolved against the filesystem, never against citations." The second is generated status: shell scripts recompute readiness, challenge, and input ledgers from the artifacts themselves, so the pipeline's state is derived from the tree instead of maintained by hand, and a stale status file cannot mislead anyone.

1. Extract108 features2. Review105 reports3. Validateadversarial pass4. Human gatequestions, challengesrulings propagate back into reports5. Study39 studies6. Executecode from task listsAppend-only knowledge basefolder locks, lessons, generated ledgers

The six-phase pipeline, coordinated through the filesystem. Every session reads the knowledge base before working and writes its lessons back after. Use "Next step" to walk through it.

The six phases: extraction, migration review, adversarial validation, the human gate of developer questions and verdict challenges, design studies, and execution. Rulings from the gate propagate back into affected reports, and every session reads from and writes back to the append-only knowledge base.

Where machines run out of authority, the pipeline queues instead of guessing. Questions only a person can answer accumulate in a structured developer Q&A file, each with the agent's suggested answer attached so the human is confirming or correcting rather than composing, and every closed answer triggers a propagation pass that updates each affected report. Design studies then turn validated verdicts into specifications behind five gates: every contributing report exists, every one has been validated, no verdict challenge is open, every governing fact has been answered by a developer or measured live, and every design fork is recorded as a decision with a default and a reversal cost. Only then is a study buildable, and buildable is a promise that the implementation phase can write the code and ask nothing.

Execution keeps the same discipline and moves it into code. One agent owns a feature from start to finish, claimed by the same atomic folder lock, in a single flat layout this time, a lesson the earlier phases paid for when tools that searched one layout silently missed the other. The study is read-only: if the design turns out to be wrong, the agent records a finding for a human to rule on instead of quietly redesigning. The study's task list is the order of work, and each task names the files it creates, edits, or deletes and the acceptance check that proves it, so every task lands as its own commit. Done means verified, not written: exercise the real path, grep for callers, because code that nothing calls is exactly the defect class the migration exists to remove, measure data against the live system rather than a stale mirror, and pass a production build. Anything half-built is listed as deferred, because an overstated done is the one failure the next agent cannot absorb.

What happened

The pipeline has now run end to end. It extracted 108 legacy features and reviewed them in 105 migration reports, 103 of which have completed the adversarial validation pass, which repeatedly overturned confident claims of exactly the kind described above. A scheduled agent routine then drafted all 39 design studies in three days: 37 cleared the buildable bar, and the other 2 are design-complete, with the few tasks still waiting on a decision named and deferred. Execution began on September 22. The shared capability studies that other features depend on went first, and within three days ten studies were implemented and verified, fourteen more were in flight, and the API test suite had grown from 878 to 1,168 tests. Features move through the phases independently, so on any given day some are shipping while others have not been claimed.

The honest tradeoffs: this approach spends machine effort lavishly to conserve human attention, and the verification is not overhead on the product, it is the product. The knowledge base compounds value but also grows, so every session pays a rising reading tax before its first useful action, and pruning it is editorial work that lands on a human. The human gates are the throughput limiter by design, and execution inherits that on purpose: an agent that finds a flawed study records a finding and keeps building rather than fixing the design itself, trading speed for a design that only changes with a person's ruling. And done is explicitly not a claim of perfection. Some features will be wrong, and the process plans for later correction instead of pretending otherwise.

The transferable lesson is that the unit of AI work should be a claim small enough to verify, produced by a session scoped small enough to audit. Execution applies the same rule to code: one task, one acceptance check, one commit, and a done that states exactly what was verified and what was deferred. Everything else, the locks, the ledgers, the adversarial pass, the gates, exists to make small claims compose into something a rebuild can actually stand on.

Next deep dive

A System Prompt Is Not a Security Boundary

Why a "never reveal this" instruction fails as a control, and how a lab makes the failure measurable: the same attack through an unprotected and a hardened pipeline, deterministic input and output guardrails, and a scorecard that weighs block rate against benign pass rate.

About a 7 minute read