Why it matters
A single prompt mixes instructions from the product team, text typed by a user and documents fetched from the internet. If all of it were flat text, the model would have no signal about which parts to obey. Roles give the model a training-time convention for precedence and give the serving stack a place to attach policy. The engineering problem is that the convention is statistical, not enforced.
Mental model
Roles are like signatures on a memo: a letter from legal outranks a sticky note on the desk. The analogy breaks because the model reads every signature as text it could imitate, so a sticky note claiming to be from legal is still just a sticky note.
How it works
- 1Define an internal message model with a small set of roles and a clear precedence order before mapping to any provider.
- 2Put durable behaviour, policy and output rules in the highest-privilege role available, and keep the text short enough to be read.
- 3Pass user text in the user role unchanged, and never concatenate it into the system string.
- 4Mark retrieved documents and tool results as data with an explicit delimiter and a role or field that is not the instruction role.
- 5Render messages through the model's chat template, then verify the exact tokens the template inserts.
- 6Test precedence with conflicting instructions in different roles and record which one the model followed.
Step 1 of 6
Define an internal message model with a small set of roles and a clear precedence order before mapping to any provider.
Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.
What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.
Text version
The page's steps: 1. Define an internal message model with a small set of roles and a clear precedence order b… 2. Put durable behaviour, policy and output rules in the highest-privilege role available, a… 3. Pass user text in the user role unchanged, and never concatenate it into the system strin… 4. Mark retrieved documents and tool results as data with an explicit delimiter and a role o… 5. Render messages through the model's chat template, then verify the exact tokens the templ… 6. Test precedence with conflicting instructions in different roles and record which one the…
- Define an internal message model with a small set of roles and a clear precedence order before mapping to any provider.
- Put durable behaviour, policy and output rules in the highest-privilege role available, and keep the text short enough to be read.
- Pass user text in the user role unchanged, and never concatenate it into the system string.
- Mark retrieved documents and tool results as data with an explicit delimiter and a role or field that is not the instruction role.
- Render messages through the model's chat template, then verify the exact tokens the template inserts.
- Test precedence with conflicting instructions in different roles and record which one the model followed.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| System role for policy | Strongest steer, stable across turns | Harder to override per request, and providers differ | The product needs per-user policy variation |
| Developer or mid-tier role | Policy scoped to one application | Support varies by provider and model | You need portability across providers |
| Everything in the user turn | Works on models without role support | No precedence signal, easy to inject | Any retrieved text can impersonate an instruction |
| Tool results in a dedicated role | Clear separation of data and instruction | Template-dependent, sometimes flattened | The serving stack collapses roles into one string |
Good for
Strongest steer, stable across turns
Cost
Harder to override per request, and providers differ
Fails when
The product needs per-user policy variation
- The product needs per-user policy variation
- You need portability across providers
- Any retrieved text can impersonate an instruction
- The serving stack collapses roles into one string
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: System role for policy is good for Strongest steer, stable across turns, costs Harder to override per request, and providers differ, and fails when The product needs per-user policy variation. Developer or mid-tier role is good for Policy scoped to one application, costs Support varies by provider and model, and fails when You need portability across providers. Everything in the user turn is good for Works on models without role support, costs No precedence signal, easy to inject, and fails when Any retrieved text can impersonate an instruction. Tool results in a dedicated role is good for Clear separation of data and instruction, costs Template-dependent, sometimes flattened, and fails when The serving stack collapses roles into one string
Failure modes
- Role flattened by the template. System and user text end up in one undifferentiated string and precedence disappears. Debug: print the rendered prompt and check that role markers appear where expected.
- User text promoted to instructions. A user message overrides policy because it was appended to the system prompt. Debug: search the codebase for string concatenation into the system field.
- Provider mismatch. A role accepted by one API is rejected or ignored by another, so the fallback behaves differently. Debug: run the same conflicting-instruction test against every provider in the fallback path.
In production
Log the role layout, not just the text, so an incident can be traced to where an instruction came from. Run a precedence test in CI whenever the model or provider changes.
Common misconceptions
People think: A system prompt is a security control.
Actually: It is a strong hint learned in training; only code-level permissions prevent an action.
People think: Roles are interchangeable labels.
Actually: Templates and training differ, and moving content between roles can change behaviour or break the request.
Interview angle
Explain roles as a learned precedence convention rendered by a template, then say why permissions must live outside the prompt. Mention provider differences and how you would test precedence. Weak answers treat the system prompt as a sandbox and cannot say what the model actually receives.
Sources
- OpenAI, Chat Completions API reference — role fields and message ordering.
- Anthropic, Messages API reference — a system parameter separate from the turn list.
- Wallace et al., The Instruction Hierarchy — training models to prioritise privileged instructions.