Why it matters
Some decisions carry consequences a model should not commit alone: a refund above a limit, a claim, a message to a customer. Adding a person seems obviously safer, but a person with a queue of approvals and no context will click through. The engineering problem is to route the right cases, supply the evidence, and measure whether the review catches anything.
Mental model
Human-in-the-loop is a second pair of eyes at the moment of commitment, not a supervisor watching the whole shift. The metaphor breaks because eyes need something to look at: without the evidence and the diff of what will change, the reviewer sees a conclusion and can only approve it.
How it works
- 1Decide which decisions need a human by consequence and irreversibility, not by feature category.
- 2Design the review packet: proposed action, evidence, alternatives considered and the effect of approving.
- 3Route by risk and confidence, and let low-risk cases pass without review.
- 4Make rejection informative by capturing the reason as a structured label.
- 5Measure reviewer agreement, time per case and the rate of reversals after approval.
- 6Feed rejected and corrected cases into evaluation and prompt changes.
Step 1 of 6
Decide which decisions need a human by consequence and irreversibility, not by feature category.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. Decide which decisions need a human by consequence and irreversibility, not by feature ca… 2. Design the review packet: proposed action, evidence, alternatives considered and the effe… 3. Route by risk and confidence, and let low-risk cases pass without review. 4. Make rejection informative by capturing the reason as a structured label. 5. Measure reviewer agreement, time per case and the rate of reversals after approval. 6. Feed rejected and corrected cases into evaluation and prompt changes.
- Decide which decisions need a human by consequence and irreversibility, not by feature category.
- Design the review packet: proposed action, evidence, alternatives considered and the effect of approving.
- Route by risk and confidence, and let low-risk cases pass without review.
- Make rejection informative by capturing the reason as a structured label.
- Measure reviewer agreement, time per case and the rate of reversals after approval.
- Feed rejected and corrected cases into evaluation and prompt changes.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Approve every action | Maximum control | Throughput collapses and reviewers rubber-stamp | Volume is high and most cases are routine |
| Review by exception | Scales, focuses attention | Misses errors the router did not flag | The confidence signal is poorly calibrated |
| Review after the fact | No latency cost | Damage is already done | The action is irreversible |
| Human as fallback only | Cheap and fast | No labels and no oversight | The failure is silent |
Good for
Maximum control
Cost
Throughput collapses and reviewers rubber-stamp
Fails when
Volume is high and most cases are routine
- Volume is high and most cases are routine
- The confidence signal is poorly calibrated
- The action is irreversible
- The failure is silent
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Approve every action is good for Maximum control, costs Throughput collapses and reviewers rubber-stamp, and fails when Volume is high and most cases are routine. Review by exception is good for Scales, focuses attention, costs Misses errors the router did not flag, and fails when The confidence signal is poorly calibrated. Review after the fact is good for No latency cost, costs Damage is already done, and fails when The action is irreversible. Human as fallback only is good for Cheap and fast, costs No labels and no oversight, and fails when The failure is silent
Failure modes
- Rubber stamp. Approval rate is near total and reversals are rare, so review adds only latency. Debug: compare approval rate with the error rate found by later sampling.
- Context-free approval. Reviewers cannot see the evidence and approve by default. Debug: audit the review screen for the fields a correct decision requires.
- Unlabelled rejection. A rejected case teaches nothing and recurs. Debug: check that rejection reasons are structured and queryable.
In production
Track approval rate, time per review, reversal rate and queue age. Alert when approval rate approaches one, which usually means the review has stopped being a control.
Common misconceptions
People think: A human in the loop makes the system safe.
Actually: Only a human with context, time and authority does; otherwise the loop adds delay and legitimacy.
People think: Human review generates training data automatically.
Actually: It does only if the reason and the corrected action are captured in a structured form.
Interview angle
Strong answers choose the review points by consequence, describe the information the reviewer needs, and name the metric that shows the review is working. Weak answers add an approval button and assume safety follows.
Sources
- Amershi et al., Guidelines for Human-AI Interaction — design guidance for control, correction and feedback.
- Google, People + AI Guidebook — patterns for human oversight, feedback and error recovery.
- NIST, AI Risk Management Framework — human oversight as a risk control with defined responsibility.