Skip to content
AI Engineer Guide.

concept · coding · intermediate · concept 13 of 30

Shell access

0 of 30 complete

Why it matters

Without execution an agent can only guess whether its code compiles or its test passes. Shell access supplies that oracle, and it also gives the model a general-purpose interface to the machine. The engineering problem is to keep the verification value while bounding what a wrong command, a hostile repository or an injected instruction can do with the same tool.

Mental model

It is handing a contractor keys to the building. The metaphor breaks because this contractor can copy the keys, read every document on the way past and leave a door open, so scope and audit matter more than trusting the person holding them.

How it works

One tool, two ledgersInteractive
15 nodes · 3 branches
validation gatestagenoterootruled out or rejected

Step 1 of 9

One tool, two ledgers

The same interface supplies the evidence and the exposure. Every control below is a compromise between keeping the exit code and closing a door.

root · current

Shell call · one command

The unit is a single command with its own exit code and output. A shell session shared across tasks loses the boundary that makes a failure replayable.

Root of the hierarchy

Also in focus

  • The oracle · what verification needs
  • The surface · what the same tool exposes
Step 1/9 · One tool, two ledgers

Figure. The shell decomposed into what verification needs and what the same tool exposes. Exit code, bounded output and a pinned working directory are the oracle; filesystem, network, process and environment are the surface, and each exposure carries the control written at the tool layer.

What to notice. The commands, paths and limits are an authored example; the page publishes no timings or byte counts, so neither is drawn. The two ledgers are the page's frame: the oracle and the attack surface are the same tool, which is why the controls live at execution rather than in the instruction. Omitted: sandbox implementation and image maintenance, which the page covers as platform work.

Text version

A shell tool drawn as two ledgers under one call. The root is a single shell call with one run id and one workspace. The oracle branch holds three gates: the exit code is surfaced as data rather than parsed from text, stdout and stderr are separated and truncated with the full log on disk, and the working directory is pinned to the task checkout in a clean environment. The attack-surface branch holds four exposures, each with a control: writes outside the workspace are checked against the resolved path and need approval, egress is allowlisted with approval for a new host, a timed-out process group is killed rather than its parent, and the environment is built without inherited credentials. A note makes the allowlist the default, one note logs the exact command line with the run id, and three crossed branches are untruncated output, parsing stdout without the exit code, and a full host shell guarded only by a careful prompt. Steps walk both ledgers and rule out each shortcut.

  1. Pin every command to the task's checkout and start it from a clean environment.
  2. Allowlist the binaries the task needs and reject shell metacharacters that compose new commands.
  3. Capture exit code, stdout and stderr separately, and truncate output at a stated line limit.
  4. Set a wall-clock timeout and kill the whole process group, not only the parent process.
  5. Log the exact command line with the run id so any failure can be replayed.
  6. Require approval for commands that write outside the workspace or open a network connection.

Trade-offs

ChoiceGood forCostFails when
Full shellExpressive, one tool for everythingLarge attack surface, hard to auditRepository is untrusted or shared
Allowlisted commandsAuditable and predictableAgent cannot compose toolsTask needs ad hoc inspection
Container per taskIsolation and reproducibilityStartup cost, image maintenanceTask needs host credentials
Host shell with approvalsFast and familiarApproval fatigue, human errorRuns are unattended
Trade-off mapInteractive

Good for

Expressive, one tool for everything

Cost

Large attack surface, hard to audit

Fails when

Repository is untrusted or shared

  1. Repository is untrusted or shared
  2. Task needs ad hoc inspection
  3. Task needs host credentials
  4. Runs are unattended

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Full shell is good for Expressive, one tool for everything, costs Large attack surface, hard to audit, and fails when Repository is untrusted or shared. Allowlisted commands is good for Auditable and predictable, costs Agent cannot compose tools, and fails when Task needs ad hoc inspection. Container per task is good for Isolation and reproducibility, costs Startup cost, image maintenance, and fails when Task needs host credentials. Host shell with approvals is good for Fast and familiar, costs Approval fatigue, human error, and fails when Runs are unattended

Failure modes

  • Output flood. A verbose test run fills the context and the agent loses the task. Debug: log output bytes per command and confirm truncation happens before the model sees it.
  • Exit code ignored. The agent reports success after a command failed because it read only stdout. Debug: assert the harness parses exit status and surfaces it in the tool result.
  • Orphan process. A timed-out command leaves a server holding a port and the next run fails. Debug: check for listeners and surviving process groups after every timeout.

In production

Monitor command duration, output size and non-zero exit rate per task. Cap concurrency, kill leftover process groups on teardown and alert on commands that touch paths outside the workspace.

Common misconceptions

People think: A sandboxed shell removes the need for approvals.

Actually: Isolation limits blast radius; it does not decide whether the command was appropriate to run.

People think: More shell output gives the agent more information.

Actually: Beyond the context budget it gives less, because the relevant lines get pushed out.

Interview angle

Treat the shell as an oracle with an attack surface. A strong answer covers allowlisting, timeouts, process groups, output truncation and how the agent learns a command failed. Saying the agent should run the tests, with no mention of exit codes or hostile repository content, leaves out the part that breaks in production.

Sources