Why it matters
Without execution an agent can only guess whether its code compiles or its test passes. Shell access supplies that oracle, and it also gives the model a general-purpose interface to the machine. The engineering problem is to keep the verification value while bounding what a wrong command, a hostile repository or an injected instruction can do with the same tool.
Mental model
It is handing a contractor keys to the building. The metaphor breaks because this contractor can copy the keys, read every document on the way past and leave a door open, so scope and audit matter more than trusting the person holding them.
How it works
Step 1 of 9
One tool, two ledgers
The same interface supplies the evidence and the exposure. Every control below is a compromise between keeping the exit code and closing a door.
root · current
Shell call · one command
The unit is a single command with its own exit code and output. A shell session shared across tasks loses the boundary that makes a failure replayable.
Root of the hierarchy
Also in focus
- The oracle · what verification needs
- The surface · what the same tool exposes
Figure. The shell decomposed into what verification needs and what the same tool exposes. Exit code, bounded output and a pinned working directory are the oracle; filesystem, network, process and environment are the surface, and each exposure carries the control written at the tool layer.
What to notice. The commands, paths and limits are an authored example; the page publishes no timings or byte counts, so neither is drawn. The two ledgers are the page's frame: the oracle and the attack surface are the same tool, which is why the controls live at execution rather than in the instruction. Omitted: sandbox implementation and image maintenance, which the page covers as platform work.
Text version
A shell tool drawn as two ledgers under one call. The root is a single shell call with one run id and one workspace. The oracle branch holds three gates: the exit code is surfaced as data rather than parsed from text, stdout and stderr are separated and truncated with the full log on disk, and the working directory is pinned to the task checkout in a clean environment. The attack-surface branch holds four exposures, each with a control: writes outside the workspace are checked against the resolved path and need approval, egress is allowlisted with approval for a new host, a timed-out process group is killed rather than its parent, and the environment is built without inherited credentials. A note makes the allowlist the default, one note logs the exact command line with the run id, and three crossed branches are untruncated output, parsing stdout without the exit code, and a full host shell guarded only by a careful prompt. Steps walk both ledgers and rule out each shortcut.
- Pin every command to the task's checkout and start it from a clean environment.
- Allowlist the binaries the task needs and reject shell metacharacters that compose new commands.
- Capture exit code, stdout and stderr separately, and truncate output at a stated line limit.
- Set a wall-clock timeout and kill the whole process group, not only the parent process.
- Log the exact command line with the run id so any failure can be replayed.
- Require approval for commands that write outside the workspace or open a network connection.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Full shell | Expressive, one tool for everything | Large attack surface, hard to audit | Repository is untrusted or shared |
| Allowlisted commands | Auditable and predictable | Agent cannot compose tools | Task needs ad hoc inspection |
| Container per task | Isolation and reproducibility | Startup cost, image maintenance | Task needs host credentials |
| Host shell with approvals | Fast and familiar | Approval fatigue, human error | Runs are unattended |
Good for
Expressive, one tool for everything
Cost
Large attack surface, hard to audit
Fails when
Repository is untrusted or shared
- Repository is untrusted or shared
- Task needs ad hoc inspection
- Task needs host credentials
- Runs are unattended
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Full shell is good for Expressive, one tool for everything, costs Large attack surface, hard to audit, and fails when Repository is untrusted or shared. Allowlisted commands is good for Auditable and predictable, costs Agent cannot compose tools, and fails when Task needs ad hoc inspection. Container per task is good for Isolation and reproducibility, costs Startup cost, image maintenance, and fails when Task needs host credentials. Host shell with approvals is good for Fast and familiar, costs Approval fatigue, human error, and fails when Runs are unattended
Failure modes
- Output flood. A verbose test run fills the context and the agent loses the task. Debug: log output bytes per command and confirm truncation happens before the model sees it.
- Exit code ignored. The agent reports success after a command failed because it read only stdout. Debug: assert the harness parses exit status and surfaces it in the tool result.
- Orphan process. A timed-out command leaves a server holding a port and the next run fails. Debug: check for listeners and surviving process groups after every timeout.
In production
Monitor command duration, output size and non-zero exit rate per task. Cap concurrency, kill leftover process groups on teardown and alert on commands that touch paths outside the workspace.
Common misconceptions
People think: A sandboxed shell removes the need for approvals.
Actually: Isolation limits blast radius; it does not decide whether the command was appropriate to run.
People think: More shell output gives the agent more information.
Actually: Beyond the context budget it gives less, because the relevant lines get pushed out.
Interview angle
Treat the shell as an oracle with an attack surface. A strong answer covers allowlisting, timeouts, process groups, output truncation and how the agent learns a command failed. Saying the agent should run the tests, with no mention of exit codes or hostile repository content, leaves out the part that breaks in production.
Sources
- Yang et al., SWE-agent — a shell-centric agent-computer interface and its guardrails.
- Wang et al., CodeAct — executable code as the action space for agents.
- Anthropic, Writing effective tools for agents — tool output design and error surfaces for agents.