Skip to content
AI Engineer Guide.

concept · concepts · beginner · concept 9 of 12

Human-in-the-loop design

0 of 12 complete

Why it matters

Some decisions carry consequences a model should not commit alone: a refund above a limit, a claim, a message to a customer. Adding a person seems obviously safer, but a person with a queue of approvals and no context will click through. The engineering problem is to route the right cases, supply the evidence, and measure whether the review catches anything.

Mental model

Human-in-the-loop is a second pair of eyes at the moment of commitment, not a supervisor watching the whole shift. The metaphor breaks because eyes need something to look at: without the evidence and the diff of what will change, the reviewer sees a conclusion and can only approve it.

How it works

Mechanism, step by stepInteractive
  1. 1Decide which decisions need a human by consequence and irreversibility, not by feature category.
  2. 2Design the review packet: proposed action, evidence, alternatives considered and the effect of approving.
  3. 3Route by risk and confidence, and let low-risk cases pass without review.
  4. 4Make rejection informative by capturing the reason as a structured label.
  5. 5Measure reviewer agreement, time per case and the rate of reversals after approval.
  6. 6Feed rejected and corrected cases into evaluation and prompt changes.

Step 1 of 6

Decide which decisions need a human by consequence and irreversibility, not by feature category.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Decide which decisions need a human by consequence and irreversibility, not by feature ca… 2. Design the review packet: proposed action, evidence, alternatives considered and the effe… 3. Route by risk and confidence, and let low-risk cases pass without review. 4. Make rejection informative by capturing the reason as a structured label. 5. Measure reviewer agreement, time per case and the rate of reversals after approval. 6. Feed rejected and corrected cases into evaluation and prompt changes.

  1. Decide which decisions need a human by consequence and irreversibility, not by feature category.
  2. Design the review packet: proposed action, evidence, alternatives considered and the effect of approving.
  3. Route by risk and confidence, and let low-risk cases pass without review.
  4. Make rejection informative by capturing the reason as a structured label.
  5. Measure reviewer agreement, time per case and the rate of reversals after approval.
  6. Feed rejected and corrected cases into evaluation and prompt changes.

Trade-offs

ChoiceGood forCostFails when
Approve every actionMaximum controlThroughput collapses and reviewers rubber-stampVolume is high and most cases are routine
Review by exceptionScales, focuses attentionMisses errors the router did not flagThe confidence signal is poorly calibrated
Review after the factNo latency costDamage is already doneThe action is irreversible
Human as fallback onlyCheap and fastNo labels and no oversightThe failure is silent
Trade-off mapInteractive

Good for

Maximum control

Cost

Throughput collapses and reviewers rubber-stamp

Fails when

Volume is high and most cases are routine

  1. Volume is high and most cases are routine
  2. The confidence signal is poorly calibrated
  3. The action is irreversible
  4. The failure is silent

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Approve every action is good for Maximum control, costs Throughput collapses and reviewers rubber-stamp, and fails when Volume is high and most cases are routine. Review by exception is good for Scales, focuses attention, costs Misses errors the router did not flag, and fails when The confidence signal is poorly calibrated. Review after the fact is good for No latency cost, costs Damage is already done, and fails when The action is irreversible. Human as fallback only is good for Cheap and fast, costs No labels and no oversight, and fails when The failure is silent

Failure modes

  • Rubber stamp. Approval rate is near total and reversals are rare, so review adds only latency. Debug: compare approval rate with the error rate found by later sampling.
  • Context-free approval. Reviewers cannot see the evidence and approve by default. Debug: audit the review screen for the fields a correct decision requires.
  • Unlabelled rejection. A rejected case teaches nothing and recurs. Debug: check that rejection reasons are structured and queryable.

In production

Track approval rate, time per review, reversal rate and queue age. Alert when approval rate approaches one, which usually means the review has stopped being a control.

Common misconceptions

People think: A human in the loop makes the system safe.

Actually: Only a human with context, time and authority does; otherwise the loop adds delay and legitimacy.

People think: Human review generates training data automatically.

Actually: It does only if the reason and the corrected action are captured in a structured form.

Interview angle

Strong answers choose the review points by consequence, describe the information the reviewer needs, and name the metric that shows the review is working. Weak answers add an approval button and assume safety follows.

Sources