Skip to content
AI Engineer Guide.

concept · concepts · beginner · concept 6 of 12

System boundaries

0 of 12 complete

Why it matters

An AI feature is rarely one system. It is a request path that touches your application, a retrieval index, one or more model providers, a tool or two, a logging pipeline and often a queue. Each hop has different failure behaviour, different latency, different data exposure and a different owner.

Without an explicit boundary you cannot answer basic engineering questions: what happens when the provider returns 503, which component is allowed to see the raw document, where does the retry happen, and who is on call when the answer is wrong? Teams that skip this step do not avoid the questions. They answer them accidentally, during incidents, with whatever code happens to run.

The boundary is also a security object. Anything crossing it can carry untrusted instructions or personal data, and the direction matters: a document coming in from a customer is untrusted input, while a prompt going out to a provider is data egress. Same edge, two different obligations.

Mental model

A factory fence. Inside, you own the machines, the inspections and the safety rails; you can open a panel and look. Outside, you have suppliers — you can specify what arrives at the gate, test it on receipt, and hold someone to a contract, but you cannot inspect how it was made.

The analogy breaks in two ways. First, the fence is logical, not geographic: a library call can move data across it without any visible gate. Second, suppliers in software change their product underneath you. A model version update is a supplier changing the alloy without telling the assembly line, which is why version pinning belongs in the boundary design.

How it works

Mechanism, step by stepInteractive
  1. 1Draw the box you operate. List the processes, data stores and queues you deploy. Everything else is external, including the model provider, the embedding API and any SaaS tool. Be literal: a vendor SDK running in your process is still a remote dependency.
  2. 2Enumerate every crossing. For each edge, record direction, payload, protocol and whether it is synchronous. A retrieval index read, a model completion, a webhook, a trace export and a document upload are five different crossings with five different risk profiles.
  3. 3Classify each crossing. Trusted or untrusted input. Contains personal data or not. Idempotent or not. Billable or not. Bounded latency or unbounded. This classification, not the architecture diagram, dictates the controls.
  4. 4Write the contract at the edge. For a model call: request schema, response schema, maximum tokens, timeout, retry policy, error mapping. For a data crossing: schema, freshness, retention. For a tool: authorisation scope, side effects, approval requirement.
  5. 5Choose failure semantics explicitly. Decide per edge whether the system fails closed (refuse), fails open (degrade to a simpler path), retries, or queues. A refusal on a legal question and a refusal on a brainstorming prompt are not the same product decision.
  6. 6Instrument the edge, not the interior. Emit one trace spanning the crossings, with per-hop latency, token counts, status and a redacted payload reference. If you can only see total latency, you cannot tell a slow retriever from a slow provider.

Step 1 of 6

Draw the box you operate. List the processes, data stores and queues you deploy. Everything else is external, including the model provider, the embedding API and any SaaS tool. Be literal: a vendor SDK running in your process is still a remote dependency.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Draw the box you operate. List the processes, data stores and queues you deploy. Everythi… 2. Enumerate every crossing. For each edge, record direction, payload, protocol and whether … 3. Classify each crossing. Trusted or untrusted input. Contains personal data or not. Idempo… 4. Write the contract at the edge. For a model call: request schema, response schema, maximu… 5. Choose failure semantics explicitly. Decide per edge whether the system fails closed (ref… 6. Instrument the edge, not the interior. Emit one trace spanning the crossings, with per-ho…

  1. Draw the box you operate. List the processes, data stores and queues you deploy. Everything else is external, including the model provider, the embedding API and any SaaS tool. Be literal: a vendor SDK running in your process is still a remote dependency.
  2. Enumerate every crossing. For each edge, record direction, payload, protocol and whether it is synchronous. A retrieval index read, a model completion, a webhook, a trace export and a document upload are five different crossings with five different risk profiles.
  3. Classify each crossing. Trusted or untrusted input. Contains personal data or not. Idempotent or not. Billable or not. Bounded latency or unbounded. This classification, not the architecture diagram, dictates the controls.
  4. Write the contract at the edge. For a model call: request schema, response schema, maximum tokens, timeout, retry policy, error mapping. For a data crossing: schema, freshness, retention. For a tool: authorisation scope, side effects, approval requirement.
  5. Choose failure semantics explicitly. Decide per edge whether the system fails closed (refuse), fails open (degrade to a simpler path), retries, or queues. A refusal on a legal question and a refusal on a brainstorming prompt are not the same product decision.
  6. Instrument the edge, not the interior. Emit one trace spanning the crossings, with per-hop latency, token counts, status and a redacted payload reference. If you can only see total latency, you cannot tell a slow retriever from a slow provider.
  7. Re-review when a component is added. Boundaries drift fastest through small additions: a new tool, a new cache, a new analytics export. Each one needs the same classification (trusted input, personal data, idempotency, billability, latency bound) and an explicit failure decision at its edge.
Inside
Your app, prompts, retrieval, schemas, retries, evaluation, logs
Outside
Model weights, provider serving, embedding API, SaaS tools
Per-edge contract
Schema, auth, timeout, retry, failure mode, cost owner
Best evidence it is wrong
An incident where nobody could name the failing hop

A concrete example

A contract-review assistant summarises clauses. The first design calls the model directly from the request handler, passes the whole document, and writes the response into the same table as the source. Three boundaries are missing: the document crosses into a third party without redaction, the call has no timeout so a slow provider holds a worker, and the summary is stored without a record of which model version produced it.

The corrected design has one service that owns all model calls, a redaction step before egress, a per-call timeout with a defined degraded response, a stored prompt and model version with every output, and a single trace ID across retrieval, redaction, inference and persistence. Nothing about the model changed. The system became diagnosable.

Trade-offs

Where the boundary sitsBuys youCosts youFails when
Calls inline in the featureFewest moving parts, fastest to buildDuplicated policy, no central budget or auditA second feature needs the same controls
Shared internal gatewayOne place for keys, retries, quotas, tracingAn extra hop and a team to own itThe gateway becomes a single point of failure without failover
Vendor-managed end to endLeast operational surfaceLittle visibility, weak portability, opaque limitsYou need per-tenant policy or a data-residency guarantee
Self-hosted model inside the fenceFull control of data and versioningGPU cost, capacity planning, serving expertiseTraffic is spiky or the team has no serving experience
Trade-off mapInteractive

Good for

Fewest moving parts, fastest to build

Cost

Duplicated policy, no central budget or audit

Fails when

A second feature needs the same controls

  1. A second feature needs the same controls
  2. The gateway becomes a single point of failure without failover
  3. You need per-tenant policy or a data-residency guarantee
  4. Traffic is spiky or the team has no serving experience

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Calls inline in the feature is good for Fewest moving parts, fastest to build, costs Duplicated policy, no central budget or audit, and fails when A second feature needs the same controls. Shared internal gateway is good for One place for keys, retries, quotas, tracing, costs An extra hop and a team to own it, and fails when The gateway becomes a single point of failure without failover. Vendor-managed end to end is good for Least operational surface, costs Little visibility, weak portability, opaque limits, and fails when You need per-tenant policy or a data-residency guarantee. Self-hosted model inside the fence is good for Full control of data and versioning, costs GPU cost, capacity planning, serving expertise, and fails when Traffic is spiky or the team has no serving experience

Failure modes

Boundary exists on the diagram only. Data leaves through logs or traces that nobody classified as egress. Debug: grep for outbound calls and inspect payloads, including telemetry, error reporters and analytics SDKs.

Retry amplification across hops. Client, gateway and provider SDK each retry, so one user retry becomes many upstream requests. Debug: count requests per user action in a trace during an injected failure; the multiplier tells you where retries are duplicated.

No degraded mode. The provider is slow and the feature is simply down. Debug: inject a fixed elevated latency and observe the user-visible state; if it is a spinner with no bound, define one.

Version drift at the edge. A provider deprecates a model and behaviour changes without a deploy on your side. Debug: record the resolved model version per response and alert when it differs from the pinned value.

In production

Put the boundary in the monitoring model: one SLO per edge, not one per feature. Provider error rate, retriever p95, tool timeout rate and end-to-end success rate should be separate series, because they fail for different reasons and are fixed by different people. Add egress allowlists so a new SDK cannot quietly open a connection, and keep a written inventory of what data crosses which edge — that document is what you hand a security reviewer, and it is also the fastest way to onboard someone onto the system.

Common misconceptions

People think: A vendor SDK inside my process is part of my system.

Actually: The SDK is your code; the service behind it is not. You cannot inspect, patch or reproduce the remote behaviour, so it belongs outside the boundary and needs a contract.

People think: Boundaries are an architecture concern, settled at design time.

Actually: They erode with every small addition. A cache, an analytics export or a debugging endpoint can create a new crossing, and the review is a recurring task.

Interview angle

System design interviews usually start with "draw the boxes" — which is a boundary question in disguise. Strong candidates state what they operate, what they call, and what they guarantee at each edge before choosing a vector database or a model. Then they name the failure semantics: timeout, retry owner, degraded response. That ordering signals production experience faster than any component name.

Start with system design questions, and expect a follow-up on data egress from the security bank.

Sources