Skip to content
AI Engineer Guide.

concept · coding · intermediate · concept 3 of 30

Architecture

0 of 30 complete

Why it matters

Two coding agents using the same model can differ by a wide margin in usefulness, and architecture explains most of the difference. One gives the model a symbol search, targeted file reads and a test runner. The other pastes whole files into a prompt and asks for a rewrite. The model is identical; the available observations are not.

Architecture also determines your incident surface. An agent with a shell is a process that can delete a database, exfiltrate a token or rewrite a lockfile. Where those capabilities live, and what stands between the model's request and their execution, is a design decision that cannot be deferred to a prompt.

Mental model

A workshop, not a worker. The bench, the tools, the safety cage around the saw, and the inspector at the end are separate fixtures. You can replace the craftsperson without rebuilding the workshop, and you can add a new guard without retraining them.

The analogy breaks on improvisation. A workshop is designed for a known set of operations; an agent's requests are open-ended, so the fixtures have to reject unknown operations safely rather than assume a fixed menu. That is why the tool layer needs a validation boundary, not just a list of buttons.

How it works

The harness around the model, layer by layerInteractive
signalbypassloop

Step 1 of 6

Assemble what it sees

Instructions, repository map, retrieved code and the transcript are merged under one token budget. The model receives this prompt and nothing else, so what the harness omits is invisible.

Active layers

  • Task + repo
  • Context builder

Hover a layer to read what it holds.

Next: The model proposes

Boundary
policy outside the prompt
Verified
by exit status
Sandbox
disposable, no secrets
Step 1/6 · Assemble what it sees

Figure. One turn of the loop, not the whole session. The model's action reaches the workspace only through the policy band and the tool layer, and the verifier reads the workspace rather than the transcript. The tool set is one band, the session store is the last, and per-tool argument schemas are omitted.

What to notice. The dashed lane is a denial: policy returns a reason and the turn ends before anything executes. The dash-dot edge is the observation going back into the transcript, which is what the next turn is assembled from. No edge runs from the model to the verifier, so a confident summary cannot mark a run verified; only an executed check with an exit status can.

Text version

A vertical stack of one agent turn. Task and repository, marked untrusted, feeds a context builder that assembles a single prompt under a budget. The prompt goes to the model behind a provider adapter, which returns one action. A policy band outside the model checks the action; an allowed action enters a sandbox group containing the tool layer and a disposable workspace copy with no ambient credentials. The workspace state goes to a verifier, which records an exit status into a diff and session record. A dash-dot edge returns the observation to the context builder, and a dashed lane carries a denial straight to the record. Six steps walk context, proposal, policy, execution, observation, and verification.

Context builder. Owns the prompt. It merges instruction files, repository structure, retrieved code and the conversation, under a token budget with an eviction policy. Keeping it separate from the loop lets you A/B test context strategies without touching control flow.

Tool layer. A small, named action space: list, read, search, patch, run. Each tool validates arguments, enforces size limits and returns structured results. The temptation is to expose a generic shell and skip the rest; that collapses tool design into prompt design and makes every failure harder to attribute.

Sandbox. The execution boundary: a container, a VM or a restricted local process with a disposable copy of the repository. It should have no ambient credentials, a CPU and wall-clock limit, and a network policy that defaults to deny. Everything the agent produces is untrusted output.

Policy. Authorisation and risk. Which paths may be written, which commands may run, which actions need approval, whether tests may be modified. Policy belongs outside the model because the model can be persuaded by content it reads; see prompt injection.

Verifier. Runs the checks that define done, then records exactly what ran and what it returned. It is a distinct component so that "verified" has a definition independent of the agent's narration.

Session store. Transcript, patches, command outputs and verification results, persisted per step. This is what makes resume, replay and review possible; without it, every failure is anecdotal.

Trade-offs

BoundarySeparate componentCostFails when
Context builderTestable context strategies, model swapsExtra abstraction to maintainThe strategy is trivial and changes weekly
Sandboxed executionContainment, reproducible environmentsStartup latency, filesystem plumbingThe task needs host resources the sandbox cannot see
Policy engine outside the promptEnforceable limits, auditable denialsPolicy must be maintained as codePolicy lags behind new tools
Independent verifierTrustworthy done signalSlower runs, duplicate workThe only available check is the agent's own output
Remote harness serviceParallel sessions, central loggingAuth, tenancy, cold startsA single developer loop is all you need
Trade-off mapInteractive

Good for

Testable context strategies, model swaps

Cost

Extra abstraction to maintain

Fails when

The strategy is trivial and changes weekly

  1. The strategy is trivial and changes weekly
  2. The task needs host resources the sandbox cannot see
  3. Policy lags behind new tools
  4. The only available check is the agent's own output
  5. A single developer loop is all you need

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Context builder is good for Testable context strategies, model swaps, costs Extra abstraction to maintain, and fails when The strategy is trivial and changes weekly. Sandboxed execution is good for Containment, reproducible environments, costs Startup latency, filesystem plumbing, and fails when The task needs host resources the sandbox cannot see. Policy engine outside the prompt is good for Enforceable limits, auditable denials, costs Policy must be maintained as code, and fails when Policy lags behind new tools. Independent verifier is good for Trustworthy done signal, costs Slower runs, duplicate work, and fails when The only available check is the agent's own output. Remote harness service is good for Parallel sessions, central logging, costs Auth, tenancy, cold starts, and fails when A single developer loop is all you need

Failure modes

  • Tool surface wider than the task. A documentation fix can run arbitrary commands because the shell is always available. Debug: log tool calls per task class; any tool that never appears in a class should not be exposed to it.
  • Observation flooding. A command returns tens of thousands of lines and evicts the task from context. Debug: measure observation bytes per call; cap at the tool boundary and write the full output to a file the agent can read selectively.
  • Verification performed by the agent. The verifier is just another prompt, so a confident summary passes. Debug: require the verification component to execute a command and record an exit status; a run without one cannot be marked verified.
  • Sandbox escape through configuration. The workspace mounts the host home directory for convenience. Debug: assert the mount list in a startup check and diff it in CI; test with a command that tries to read a known secret path.
Failure modes, one at a timeInteractive

Symptom

A documentation fix can run arbitrary commands because the shell is always available.

Debug step

Log tool calls per task class; any tool that never appears in a class should not be exposed to it.

Figure. Each failure with its symptom and the check that would confirm it.

What to notice. A failure without a debug step is a rumour. Pick the symptom you are seeing; the check is the next action.

Text version

Failure modes: Tool surface wider than the task: A documentation fix can run arbitrary commands because the shell is always available. Debug: log tool calls per task class; any tool that never appears in a class should not be exposed to it. | Observation flooding: A command returns tens of thousands of lines and evicts the task from context. Debug: measure observation bytes per call; cap at the tool boundary and write the full output to a file the agent can read selectively. | Verification performed by the agent: The verifier is just another prompt, so a confident summary passes. Debug: require the verification component to execute a command and record an exit status; a run without one cannot be marked verified. | Sandbox escape through configuration: The workspace mounts the host home directory for convenience. Debug: assert the mount list in a startup check and diff it in CI; test with a command that tries to read a known secret path.

In production

Latency is dominated by model turns, not by tool execution, so the harness should stream events and let the UI render progress. Cold start matters for short tasks: a container that takes longer to boot than the task takes to run will dominate p50.

Instrument per component: context tokens per turn, tool latency by name, policy denial rate, verification pass rate, and steps to completion. Component-level metrics turn "the agent got worse" into "reads doubled because the repo map stopped indexing a new package". Keep model-specific code behind the adapter; provider migrations are routine, and a harness that assumes one message schema pays for it every time.

Common misconceptions

People think: The model does the work, so architecture matters less than model choice.

Actually: The same model with symbol search, targeted reads and a real test runner behaves very differently from one with a file dump and a chat box. The harness supplies the observations the model reasons over.

People think: A sandbox is only needed for untrusted users.

Actually: The agent reads untrusted content: issues, dependencies, test fixtures, web pages. A prompt injection can arrive through any of them, so the sandbox protects against the agent's own inputs.

Interview angle

A common system-design prompt is "design a coding agent for a monorepo". Strong answers draw the components and then defend the boundaries: why the verifier is separate, why policy is not a prompt, why the sandbox is disposable. Interviewers push on failure isolation and on what happens when the model asks for something the harness forbids. See agentic coding interview questions, sandboxing and traces.

Sources