Skip to content
AI Engineer Guide.

concept · coding · intermediate · concept 14 of 30

TDD loops

0 of 30 complete

Why it matters

Agents optimise for output that looks plausible, and plausible code can satisfy a prompt while missing the actual defect. Test-first development supplies an oracle before the implementation exists, which turns an open-ended generation into a search with a stopping condition. It also limits collateral edits, because any change that does not move the failing test is unnecessary by construction. Reviewing the test diff is part of the loop, because the agent controls both sides of the oracle and can weaken the assertion it is meant to satisfy.

Mental model

It is cutting a lock and key together: the test is the key and the code is the lock. The metaphor breaks because the agent can file the key down until any lock opens, which is why the test diff needs its own review rather than trust in the loop.

How it works

Mechanism, step by stepInteractive
  1. 1Turn the report into one assertion about observable behaviour with the smallest input that triggers it.
  2. 2Run the test on the base commit and capture the failure message as evidence.
  3. 3Allow changes to production code only until that test passes, then stop the edit loop.
  4. 4Run the surrounding package tests to catch damage outside the targeted behaviour.
  5. 5Review the test diff as carefully as the code diff, looking for weakened assertions.
  6. 6Add a second test only for a distinct behaviour, never for another branch of the same one.

Step 1 of 6

Turn the report into one assertion about observable behaviour with the smallest input that triggers it.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Turn the report into one assertion about observable behaviour with the smallest input tha… 2. Run the test on the base commit and capture the failure message as evidence. 3. Allow changes to production code only until that test passes, then stop the edit loop. 4. Run the surrounding package tests to catch damage outside the targeted behaviour. 5. Review the test diff as carefully as the code diff, looking for weakened assertions. 6. Add a second test only for a distinct behaviour, never for another branch of the same one.

  1. Turn the report into one assertion about observable behaviour with the smallest input that triggers it.
  2. Run the test on the base commit and capture the failure message as evidence.
  3. Allow changes to production code only until that test passes, then stop the edit loop.
  4. Run the surrounding package tests to catch damage outside the targeted behaviour.
  5. Review the test diff as carefully as the code diff, looking for weakened assertions.
  6. Add a second test only for a distinct behaviour, never for another branch of the same one.

Trade-offs

ChoiceGood forCostFails when
Strict test firstClear stopping conditionSlow start when behaviour is hard to assertChange is exploratory or UI-only
Characterisation test of current behaviourSafe refactorsLocks in existing bugsCurrent behaviour is itself wrong
Agent writes test and codeFastSelf-confirming testsCorrectness matters
Snapshot testsCheap coverageBrittle and reviewed as noiseOutput is non-deterministic
Trade-off mapInteractive

Good for

Clear stopping condition

Cost

Slow start when behaviour is hard to assert

Fails when

Change is exploratory or UI-only

  1. Change is exploratory or UI-only
  2. Current behaviour is itself wrong
  3. Correctness matters
  4. Output is non-deterministic

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Strict test first is good for Clear stopping condition, costs Slow start when behaviour is hard to assert, and fails when Change is exploratory or UI-only. Characterisation test of current behaviour is good for Safe refactors, costs Locks in existing bugs, and fails when Current behaviour is itself wrong. Agent writes test and code is good for Fast, costs Self-confirming tests, and fails when Correctness matters. Snapshot tests is good for Cheap coverage, costs Brittle and reviewed as noise, and fails when Output is non-deterministic

Failure modes

  • Test written to match the patch. The assertion encodes the implementation including its bug. Debug: check out the base commit, apply only the test and confirm it fails.
  • Assertion weakening. The agent loosens an existing assertion until it passes. Debug: diff test files and flag removed conditions or broadened matchers.
  • Green on the wrong suite. The targeted test passes but the package suite was never run. Debug: record which command produced the green result and re-run the wider suite.

In production

Track tests added per task and mutation score on changed files. Reject patches that modify assertions without a recorded failing-first run.

Common misconceptions

People think: TDD means the agent writes tests, so the tests are trustworthy.

Actually: Tests written after the fix tend to encode the fix, so provenance matters more than presence.

People think: A passing suite means the feature works.

Actually: It means the assertions you have hold; coverage of the new behaviour is the open question.

Interview angle

Anchor the loop on a test that fails for the right reason before code changes. A strong answer reviews the test diff as carefully as the code and mentions mutation testing or assertion checks. Describing test-first as a prompt instruction, where the agent writes both sides and nobody inspects the assertions, is the weak answer.

Sources