Why it matters
Availability targets alone cannot describe an assistant that answers every request quickly and incorrectly. A quality indicator requires a labelled sample, a judge with known agreement, or a deterministic check such as citation validity, and each has a cost and a lag. The objective exists to make the trade explicit: how much wrongness is acceptable, measured over what window, and what the team does when the budget is spent.
Mental model
An objective is a thermostat setting: it does not heat the room, it decides when the heating turns on. The analogy breaks because a thermostat measures one number, while a good AI service needs several targets that can conflict, such as speed against depth of reasoning.
How it works
The interactive figure loads with JavaScript. The text version below describes the same frame.
Figure. A modelled latency distribution cut at a 2 s objective, with the requests past the cut consumed from a 30-day error budget. The distribution is log-normal and every number is generated by the stated model; the shape — a thin tail that widens with load — is the claim.
What to notice. At ×1 traffic 1.7% of requests miss the 2 s objective and a 2% budget survives the month. At ×2 traffic 8.0% miss it, p95 has crossed 2 s, and the budget is spent by day 8. The density's peak barely moves while its tail crosses the line: the objective fails before the typical request feels slow.
Text version
A density curve of request latency from 0 to 6,000 ms with a vertical 2,000 ms SLO line. The area to its right is filled red, labelled requests over the SLO, and annotated 1.7% of requests past this line; the rest of the curve is teal. Both axes are labelled: latency in milliseconds across the bottom, share of requests per 200 ms bucket up the left. Below, a second panel plots the error budget remaining from 100% at day 0 to 17% at day 30 on its own day axis. A traffic slider from ×1 to ×2.5 redraws both panels: at ×2 the p95 is 2,320 ms, the annotation reads 8.0% of requests past this line, and the budget reaches zero on day 8. Three statistics under the legend report p95, the over-SLO share and how long the 2% budget lasts.
- Pick the user journey and the moment of pain, then define the event that counts as good for it.
- Write the indicator as good events divided by valid events, and specify what excludes a request from the denominator.
- Set the target from measured behaviour and user tolerance, not from an aspiration.
- Add a quality indicator with a stated sampling rate, grader and agreement check against human labels.
- Attach an error-budget policy that says what freezes when the budget is exhausted.
- Alert on burn rate over a short and a long window, so both sudden outages and slow degradation page someone.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Latency target only | Easy to measure and alert | Misses wrong but fast answers | The product's value is answer correctness |
| Quality target from human labels | Trustworthy | Expensive and delayed | Traffic changes daily |
| Quality target from a model judge | Cheap and fast | Judge drift and bias | The judge was never validated on your domain |
| Strict target | Protects users | Budget exhausts and blocks releases | The system is still stabilising |
Good for
Easy to measure and alert
Cost
Misses wrong but fast answers
Fails when
The product's value is answer correctness
- The product's value is answer correctness
- Traffic changes daily
- The judge was never validated on your domain
- The system is still stabilising
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Latency target only is good for Easy to measure and alert, costs Misses wrong but fast answers, and fails when The product's value is answer correctness. Quality target from human labels is good for Trustworthy, costs Expensive and delayed, and fails when Traffic changes daily. Quality target from a model judge is good for Cheap and fast, costs Judge drift and bias, and fails when The judge was never validated on your domain. Strict target is good for Protects users, costs Budget exhausts and blocks releases, and fails when The system is still stabilising
Failure modes
- Unvalidated judge. The quality indicator reports success while users complain. Debug: compare judge verdicts with a human-labelled sample and report agreement per category.
- Denominator drift. A change in what counts as a valid request makes the ratio jump without any behaviour change. Debug: chart numerator and denominator separately and diff the exclusion rule in version control.
- Budget without policy. The budget is exhausted and nothing changes, so the target becomes decorative. Debug: check that a burn-rate alert has an owner and a documented action.
In production
Review objectives quarterly against actual user complaints, and keep the sampling cost of quality measurement visible in the budget. Page on burn rate, report on budget remaining, and record every exception to the policy.
Common misconceptions
People think: An objective is a promise to users.
Actually: It is an internal operating target; the external promise is usually looser and stated in a contract.
People think: Quality cannot be a target because it is subjective.
Actually: It can be measured as a rate over a defined judgement, provided the judgement is written down and its agreement is checked.
Interview angle
Define the good-event ratio, add a quality indicator with a stated grader, and attach a budget policy to the number. Explain multi-window burn alerts. An availability percentage with no numerator, denominator or consequence is not an objective.
Sources
- Google, SRE Workbook: Implementing SLOs — indicator definitions, targets and error-budget policy.
- Google, SRE Book: Service Level Objectives — the relationship between indicator, objective and agreement.