Stop reading every line of AI-generated code

Nobody wants to read a 5,000-line diff. But that is what lands in the pull request when an agent spends an afternoon refactoring a legacy service or scaffolding a new Laravel module end to end.

The failure mode that follows is worth naming precisely, because it does not look like failure while it is happening. An agent scaffolds a multi-tenant module in one pass — migrations, models, policies, jobs, tests. The diff is correct, mostly. You spend two evenings reading it line by line and approve it. Weeks later something breaks in production: a queued job that ran outside the tenant scope. You read that file. You read that exact method. Your eyes went over it and your brain filed it as “looks like the other ones.”

It feels like diligence. It is theater.

Where line-by-line review actually breaks

Code review guidance has been consistent for two decades: reviewers find defects reliably in chunks of roughly 200 to 400 lines, and effectiveness drops sharply past that. Nothing about that number has changed because we bolted an LLM onto the front of the process. Human attention is the constant. The volume of code is the variable, and it just went up by an order of magnitude.

Three things go wrong when you push past that limit:

• You review syntax, not semantics. Naming, formatting, and shape are cheap to evaluate, so tired brains drift toward them. A perfectly formatted diff can still ship an N+1 query, a missing index, or a broken state machine.
• You lose the cross-file thread. The bug is almost never in the file you are reading. It is in the interaction between the job, the observer, and the tenant scope — three files you looked at twenty minutes apart.
• You pattern-match against yourself. Agent-generated code is internally consistent. That consistency reads as correctness. Every file looks like the last one, so your reviewer instinct stops firing.

None of this is new. What changed is that we used to write code slowly enough that review capacity roughly matched output. That coupling is gone. Generation scales with tokens; reading scales with eyeballs.

What agents genuinely do better — and what they don’t

A review agent pointed at a repository does not read files the way you do. Given the right tooling it walks the AST, follows call graphs, and traces a value from a controller through a service into a model and out to an external API. It does that for file forty with the same attention it gave file one.

That is the real advantage: breadth and consistency, not intelligence. An agent will happily check all 340 controller actions for a missing authorization call. You will check the twelve you remember.

I want to be equally clear about the other side, because “agents review better than humans” is a bad summary of what I actually believe:

• Agents hallucinate findings. They will confidently flag a race condition that cannot occur, and the fix they suggest can be worse than the code.
• Agents have no idea whether the feature should exist. Product intent is not in the diff.
• Agents are weak on the thing that only shows up at runtime under load — the query that is fine with 200 rows and fatal with 2 million.
• An agent reviewing code written by the same model family shares its blind spots.

So the move is not “trust the agent instead of your eyes.” It is: give the agent the work that scales with volume, and keep for yourself the work that scales with judgment. That split is the same one I drew in AI vs traditional development — the boundary moved, it did not disappear.

The shift: own the spec, not the syntax

If you are not reading the code, something else has to hold the line. For me that is the specification layer — and writing it is now the highest-leverage thing I do on an agent-assisted project.

Concretely, that means:

• Executable acceptance criteria. Not a Notion doc. Feature tests that fail before the agent starts and pass when it is done.
• Property-based tests for anything with arithmetic or state. Pricing, proration, inventory, refunds. Instead of three examples, assert the invariant: a refund never exceeds the captured amount, total allocated inventory never exceeds stock on hand.
• Contract tests at every boundary. Webhook payloads, third-party API responses, queue message shapes. This is where agent code fails quietly, because the happy path is the only path in the training data.
• Machine-checkable rules for the things I keep repeating. Static analysis at a strict level, an architecture-test suite asserting “no job may run without a tenant context,” a lint rule for direct DB access outside repositories.

That last one is the real trick. Every review comment I would type twice becomes a rule instead. If the agent can violate an invariant and the pipeline stays green, that is my bug, not the agent’s. The numbers behind why this layer is non-negotiable — and the CI configuration I run — are in test AI code too.

When a check fails, I do not hand-patch the file. I fix the prompt, tighten the spec, or add the missing rule, then re-run. That loop is the practical core of agentic workflows, and it only works if the feedback the agent receives is precise. Vague context produces vague code — the point I made in prompt engineering vs context engineering.

A multi-agent review pipeline that earns its cost

One giant “review this PR” prompt produces mush. Narrow agents with narrow briefs produce findings you can act on. This is the verify stage of the plan, execute, verify, deploy loop, and it deserves its own agents for the same reason it deserves its own step. The setup I run now has three, and they all execute inside CI rather than in my editor:

1. Correctness. Runs the suite, static analysis, and the architecture tests. Then reviews the diff strictly against the written acceptance criteria — its brief explicitly forbids style commentary. Output: does this satisfy the spec, and where does it not.
2. Security. Injection paths, authorization gaps on new routes, secrets in config, unsafe deserialization, mass-assignment on new models, anything touching a webhook signature. This one gets the OWASP-shaped checklist and nothing else.
3. Regression. Diffs behavior against the previous stable build. It reads the migration files, flags schema changes without a rollback path, hunts for queries added to hot code paths, and cross-references the touched files against past incident history.

Each agent needs real access to be useful — the schema, the logs, the actual test output, not a pasted snippet. That is exactly the problem MCP servers solve, and it is the difference between a reviewer that guesses and one that checks.

Two rules keep this from becoming noise. First, findings are severity-ranked and anything below “would break in production” goes to a comment, never a block. Second, a second agent tries to refute each finding before I see it. A large share of the initial findings die there, and they are exactly the ones that would otherwise burn an afternoon.

The loop

The pipeline is only worth building if it closes without me in the middle of it:

[Spec + tests] --> [Agent generates] --> [Static analysis + type checks]
                          ^                          |
                          |                          v
                   [Self-correction] <-- [Test suite + review agents]
                          |
                          v (all green)
                   [Human: seams review] --> [Merge / deploy]

Failures feed back into the agent’s context automatically. I only see the branch once it is green, which is the same discipline that makes CI and CD work in the first place — the pipeline is the gate, not the reviewer’s inbox.

What I still read myself

“Stop reading code” is a slogan, and taken literally it is wrong. Here is my actual list of things I open every time, no matter how green the pipeline is:

• Migrations. Anything irreversible or lock-heavy on a large table.
• Auth and permission boundaries. Policies, scopes, middleware, anything that decides who sees what.
• Money paths. Charges, refunds, discounts, tax, billing webhooks. Tests are necessary here and not sufficient.
• New architectural seams. A new queue, a new external dependency, a new cache layer. The code may be fine; the decision may not be.
• Anything the agent changed that nobody asked it to change. Unrequested scope is the loudest smell in agent diffs.

That list is maybe 3% of a large diff. Reading 3% carefully beats skimming 100%, and it is the part where my decade of production scars actually transfers. The rest — the 340 controller actions, the DTO fields, the test scaffolding — is machine work, and I stopped pretending otherwise.

Takeaways

• Line-by-line review does not scale to agent-sized diffs. Past a few hundred lines you are performing diligence, not doing it.
• Use agents for breadth, not judgment. Cross-file tracing and repeated checks are their edge; product intent and architectural taste are not.
• Convert review comments into rules. Anything you would say twice belongs in a test, a static-analysis level, or an architecture test.
• Split the review agent into narrow briefs — correctness, security, regression — and make a second pass try to refute each finding before it reaches you.
• Keep a short human read list: migrations, auth, money, new seams, unrequested changes.

The uncomfortable question is what your team’s review process is actually for. If it exists to catch defects, most of it should be automated by now. If it exists to spread knowledge, say that out loud and redesign it around that goal instead.

Where does your review process still depend on someone reading every line — and what would it take to turn that into a check? Tell me via contact, I collect these. 🤘