Large-Scale Code Migrations with LLMs

Since I joined Shopify, one of the recurring challenges has been migrating the monolith to newer APIs and DSLs. When I introduced job-iteration, my team had to convert most job classes from def perform to the Iteration API. When we upgraded from Rails 4 to Rails 5, dropping all attr_accessible usage took four months. Same for test DSL changes, validation frameworks, GraphQL schema patterns—any API shift at scale.

The pattern is always the same: you either keep the old API forever or push a herculean effort to update thousands of call sites. We still have about 1,000 spots using an input validation DSL we deprecated two years ago. Product needs to ship, so nobody prioritizes the cleanup.

The Old Playbook

In a large org, you have two options for code migrations. Staff a small team to grind through all the changes, or enforce burn-down charts for each code owner to migrate their slice. The second approach requires documentation, convincing managers of the importance, and accepting that it will take a while.

A tiny subset of migrations can be done with static analysis. But most real migrations involve context: understanding what the code does, how to preserve behavior, what the new pattern should look like in this specific case.

What LLMs Change

Coding agents have shifted this landscape. You no longer have to grind through 1,000 PRs as a small group or negotiate with every team to stop using that deprecated API.

Define a clear set of steps, provide few-shot examples, establish the rules — and agents can do most of the work. You still review every change, but if 80% of the output is correct on the first pass, you've saved enormous effort.

The Tooling Gap

Here's the problem: you can't ask Claude Code to "find all usage of X and migrate to Y" when there are 500 call sites. To start with, agents today lack persistent, long-lived task lists. Many startups focus on assigning GitHub issues to agents, and sure, you could create 500 issues each with a prompt — but that's tedious and inflexible. Your v1 prompt will have 30% success rate. You'll iterate, and v2 will hit 70%. Updating 500 issue descriptions isn't realistic.

Sometimes you need to enrich the input before any work can happen. In a recent migration, I had to iterate over 300 GraphQL object definitions in Ruby. But to act on each, I needed to know which database table that object was based on — which meant matching each GraphQL type to its underlying ActiveRecord class.

So before the actual migration, I spawned 300 Claude instances via xargs just to collect that extra data. Each instance wrote its result to a file. Except multiple Claude instances calling WriteFile operation on the same file doesn't work, so I had to instruct it to use echo result >> out.csv for filesystem-level appends. Then the real migration work happened—also orchestrated via xargs -P for parallelism.

# Enrichment phase: collect metadata
cat graphql_types.txt | xargs -P 20 -I {} bash -c \
  'claude --print "For GraphQL type {}, find the ActiveRecord model it wraps and echo the table name" >> enrichment.csv'

# Migration phase: actual changes
cat enrichment.csv | xargs -P 10 -I {} bash -c \
  'claude "Migrate GraphQL type {} using enrichment data: {}"'

This works, but it's held together with shell scripts and hope.

What's Missing

An orchestration layer for agent-driven migrations would need:

1. Persistent task queues — a list of 500 items that survives across sessions
2. Prompt iteration — update the prompt template without recreating every task
3. Enrichment pipelines — collect data in phase one, feed it to phase two
4. Progress tracking — which items succeeded, which failed, which need retry with different approach

None of the current agent tooling provides this. You're left stitching together xargs, CSV files, and bash loops.

The Shift

Large-scale code migrations used to require thousands of human hours and organizational coordination. Agents have changed the economics: if you can formulate the migration as clear rules with examples, you can parallelize the execution across hundreds of agent instances.

The tooling is primitive and you'll write shell scripts and deal with filesystem append semantics. But the fundamental constraint has shifted from "we don't have enough engineers" to "we haven't built the orchestration layer yet."

If you've built tooling for this, I'd like to hear about it.