Code sandbox

How a crew gets exact counts, sums, and sorts by running short programs in the code sandbox, how each run is contained, and how to run its pod under a sandboxed runtime such as gVisor.

Language models are unreliable at counting, summing, sorting, and filtering, especially small local models reading a long listing. Asked how many pods are running in each namespace, a model gives a confident, plausible, and often wrong number. Kubemoot does not ask the model to do that work. It gives the work to a program.

The code sandbox is an MCP server that runs short Python or bash programs for the compute agent and returns what they print. The models decide what to compute. The sandbox makes the numbers real.

Why it exists

In a crew, arithmetic goes to the compute agent. Another agent gathers the raw data. The compute agent writes a short program that reads it and computes the answer. The sandbox runs the program and returns the exact output, the compute agent reports that result, and the coordinator uses it.

This is the compute contract, and the runtime enforces it: the compute agent may not state a number that did not come from running code. A count that the model read off a list is not an answer. A count a program printed is.

How data reaches it

The container that runs programs has no network path to the places data lives, so the data comes to it:

  1. An agent fetches the data from the live system, and each tool result is stored as its own file in the crew’s workspace, announcing it on the bus.
  2. The workspace-access materializer, a sidecar in the sandbox pod, reads the referenced objects from the store over the message bus and writes them to a shared volume under /kubemoot/workspace.
  3. The compute agent’s program reads those files and computes over them locally.

The materializer is the only part of the pod that talks to the bus. The sandbox container that runs programs holds no bus credentials and talks to nothing itself. Programs see only /kubemoot/workspace. Data goes in the program’s code or in a file there, never through stdin.

How a run is contained

The sandbox runs code a model wrote, so every run is locked down.

  • No credentials and no Kubernetes access. The pod holds no secrets and no service account token. It only computes over data already handed to it. The execute_code description tells the model so: no cluster API and no kubectl, so a crew that needs live data selects the agents that fetch it.
  • Limited egress. With network policies on, the crew’s policy limits the pod’s outbound traffic to the message bus, which the materializer sidecar uses to stage artifacts. The pod has no route to external services.
  • A fresh temporary directory. Each run starts in a new directory, which is also its HOME, and the directory is removed afterwards.
  • No stdin. A read returns end of input at once, so a program that expects piped data fails fast instead of waiting.
  • Its own process group. At the time limit the whole group is killed, and so is anything a finished program left running in the background.
  • Capped output. Stdout and stderr are each capped, and the result says when output was cut.
  • A small image. Python with the standard library only, bash, and an init process that reaps what a killed program leaves behind.

A program that fails is a normal result with its exit code. Only an unknown language or a program that cannot start is a tool error. At worst, a bad program wastes the pod’s CPU until the time limit ends it.

Tools and settings

ToolInputResult
execute_codelanguage (python or bash), code, optional filenamestdout, stderr, exit code, execution time
validate_codethe samewhether the syntax is valid, and the errors if not
SettingDefaultMeaning
EXECUTION_TIMEOUT30Seconds a program may run
MAX_OUTPUT_BYTES65536Cap on each of stdout and stderr
SANDBOX_TMPDIRthe OS defaultWhere run directories are created
MCP_TRANSPORTstdiostdio or http. Use http for new installs.
PORT3000The listening port when the transport is http; an invalid value stops the server at start

These are environment variables on the sandbox container. The component reference is the code-sandbox README.

Isolation: run it under a sandboxed runtime

The containment above limits what a program can reach through the pod. It does not change the fact that the program runs on the node’s kernel. A runtime such as gVisor puts a user-space kernel between the pod and the host. With it, model-written code no longer reaches the host kernel directly.

The sandbox is an MCPServer, so it takes the same optional field as any MCP server, spec.runtimeClassName:

apiVersion: kubemoot.ai/v1alpha1
kind: MCPServer
metadata:
  name: code-sandbox
spec:
  # image and the workspace-access sidecar as in your crew
  transport: http
  port: 3000
  healthPath: /healthz
  readinessPath: /readyz
  env:
    - name: MCP_TRANSPORT
      value: http
  runtimeClassName: gvisor

Under a sandboxed runtime the sandbox must use transport: http. In stdio mode the operator adds the mcp-bridge sidecar, which talks to the server over named pipes in a volume the two containers share. gVisor does not share those pipes between containers, so the bridged sandbox crash-loops. With http the sandbox serves MCP itself over Streamable HTTP at /mcp, the operator adds no bridge, and the gateway calls it directly. The same applies to any MCP server: third-party stdio servers behind the bridge do not run under gVisor without node-level containerd configuration, so run sandboxed only servers that speak HTTP.

Setting it is optional. Left unset, the pod carries no runtimeClassName and runs under the node’s default runtime, so a plain kind cluster with default settings keeps working with no extra install. To use a sandboxed runtime, the cluster owner installs it on the nodes that will run the sandbox and creates a RuntimeClass whose handler names it. Kubemoot installs neither. Isolation covers the install steps and the same field for Agents.

When the class is missing

If the MCPServer names a class that does not exist, the operator checks before it builds anything:

ConditionMeaning
RuntimeClassAvailable=TrueThe class exists. The Deployment is built with it.
RuntimeClassAvailable=False, reason RuntimeClassMissingThe class does not exist. The operator holds the Deployment, sets the MCPServer phase to Pending, and puts the missing name in the condition message.
absentThe MCPServer asks for no class.

The fix is on the cluster: install the runtime and create the RuntimeClass, or clear runtimeClassName. The operator watches RuntimeClass objects, so the server proceeds the moment the class appears, with no retry timer. A Deployment that already exists is left as it is if the class is deleted while it runs.

The compute agent has no sandbox to call while the server is held, so a crew that asks for a class its cluster does not have cannot compute until the class exists. Check RuntimeClassAvailable first when the compute agent reports that its tool is unavailable.

These controls are one part of how a crew resists injected instructions. The Crew threat model shows where each one acts and what remains undefended.