Skip to content
AI Engineer Guide.

concept · coding · intermediate · concept 25 of 30

Code review

0 of 30 complete

Why it matters

Review is the bottleneck once agents can produce patches faster than people can read them. A model can scan a diff in seconds and catch missing null checks, unused imports and inconsistent error handling. It cannot know why the change was made or which caller depends on the old behaviour. The value depends on routing each kind of finding to the mechanism that can actually resolve it.

Mental model

It is a second pair of eyes that was not in the room. The metaphor breaks because those eyes will confidently describe a conversation they never heard, and their confidence reads the same as the confidence of someone who was there.

How it works

Twenty-four findings, four routesInteractive
13 nodes · 5 branches
  • 24
    • 14
    • 6
    • 3
    • 1
stagevalidation gaterootfilenoteruled out or rejectedbar = findings · child counts sum to the parent · scaled to the diff (24)

Step 1 of 7

Route by who can settle it

Four classes, 24 findings: 14 are the linter's, 6 can be settled from a line, 3 need a caller outside the diff, and 1 needs the intent. The split is the review design, and the counts are what make it arguable.

root · current

One diff · 24 findings

The reviewer gets the diff, the task and the relevant test output. Not the repository: a reviewer that reads everything spends the same attention per file as a human, with less ability to know what matters.

Root of the hierarchy · 24 counted

Also in focus

  • Mechanical · settled by a tool
  • Local · settled from a line
  • Cross-file · needs a caller
  • Intent · a person decides
Step 1/7 · Route by who can settle it

Figure. One diff's 24 findings split by who can settle them: 14 the linter already knows, 6 checkable from a changed line, 3 that need a caller outside the diff, and 1 that needs the intent behind the change. The crossed branch is style restated as a comment, which duplicates the linter and teaches the reader to skim.

What to notice. The diff, the findings and the counts are an authored example; the page publishes no review measurements, so the counts are one sample and not a rate. The tree is a routing rubric, not a severity ranking: each class moves to the mechanism that can settle it, and only intent needs a human verdict, while severity escalates a finding without rerouting it. Omitted: comment wording and review latency, which the page treats as review craft rather than as routing.

Text version

A review rubric drawn as a tree with counts. The root is one diff with 24 findings, with the task description and test output attached. Four routes hang from it, and their counts sum to 24. Mechanical has 14, settled by the linter, semgrep or typecheck, with a gate that drops duplicates before posting and a crossed style finding that repeats the formatter. Local and checkable has 6, settled from a changed line, with a gate requiring file, line, severity and a failure scenario, and an example at refund.ts line 41 marked high severity. Cross-file consequence has 3, needing a caller outside the diff, with a gate that reads call sites or asks instead of approving. Intent and architecture has 1, where the agent states options and a person decides. A crossed branch rules out an architectural verdict from the diff alone. A final note counts acceptance per rule so a noisy rule can be retired. Steps route by class, deduplicate against the linter, ground local findings, require call sites, hand intent off, rule out the architectural verdict, and retire the noisy rule.

  1. Give the reviewer the diff, the task description and the relevant test output rather than the whole repository.
  2. Ask for findings with file, line, severity and a concrete failure scenario.
  3. Drop any finding that cannot point at a changed line or a missing test.
  4. Run deterministic checks first so the model does not spend attention on what a linter already reports.
  5. Route high-severity findings to a human and record whether each was accepted.
  6. Track acceptance rate per rule and retire rules that mostly produce noise.

Trade-offs

ChoiceGood forCostFails when
Agent review before humanCheap first pass, fast feedbackFalse positives, reviewer fatigueChange is architectural
Agent review after approvalCatches last-minute slipsSlows mergeTeam needs fast turnaround
Diff-only reviewFocused and cheapMisses cross-file consequencesChange alters a shared interface
Repository-wide reviewBroader contextCost and noiseDiff is large
Trade-off mapInteractive

Good for

Cheap first pass, fast feedback

Cost

False positives, reviewer fatigue

Fails when

Change is architectural

  1. Change is architectural
  2. Team needs fast turnaround
  3. Change alters a shared interface
  4. Diff is large

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Agent review before human is good for Cheap first pass, fast feedback, costs False positives, reviewer fatigue, and fails when Change is architectural. Agent review after approval is good for Catches last-minute slips, costs Slows merge, and fails when Team needs fast turnaround. Diff-only review is good for Focused and cheap, costs Misses cross-file consequences, and fails when Change alters a shared interface. Repository-wide review is good for Broader context, costs Cost and noise, and fails when Diff is large

Failure modes

  • Confident false positive. The reviewer flags a bug the surrounding code already handles. Debug: check whether the comment cites a line outside the diff and read that file.
  • Style noise. Review is dominated by formatting the linter would catch. Debug: compare findings against linter output and remove duplicates.
  • Missing context. The reviewer approves a change that breaks a caller it never saw. Debug: ask for the call sites of each changed signature and verify they were read.

In production

Track findings per change, acceptance rate and review latency. Cap review tokens per diff and skip review for generated or vendored files.

Common misconceptions

People think: An agent reviewer replaces human review.

Actually: It shifts attention: humans decide intent and risk while the agent checks mechanics.

People think: More comments means a better review.

Actually: Unverifiable comments cost reviewer time and reduce trust in the next review.

Interview angle

Split mechanical checks from judgement and require line-grounded findings. A strong answer measures acceptance rate and explains how the agent changes human attention rather than replacing it. Proposing that the reviewer replaces people, with no triage or false-positive cost, is the weak answer.

Sources