Harness engineering: give every sub-agent a memory, CPU and time budget
An agent that impresses in a demo breaks in production when one sub-agent takes all of the host's memory or runs forever. The harness must give each sub-agent an execution contract, detect overruns and decide: retry or stop. Method, Docker options and their limits, and results from an internally reproducible example, limits included.
Transparency note. This article follows IgnitionAI's editorial policy. Technical facts link to their primary documentation, consulted on 11 November 2026. Figures come from an example built by IgnitionAI, reproducible internally, measured on 11 November 2026. Recommendations are IgnitionAI's opinions. IgnitionAI builds AI agents: this is a conflict of interest. Salim Laimeche is co-author of Building Agentic AI Apps with Next.js and TypeScript (Packt), available for pre-order, due December 1, 2026.
An agent that succeeds in a demo can fail in production for a mundane reason: one sub-agent takes all of the host's memory, or runs forever. The model is rarely to blame. What is missing is an execution contract for each sub-agent, and a harness that enforces it.
Anthropic defines an agent harness as "the system that enables a model to act as an agent: it processes inputs, orchestrates tool calls, and returns results" (Demystifying evals for AI agents). This article covers one of its jobs: bounding what each sub-agent can consume.
In short
- Give each sub-agent a contract: memory, CPU, process count, duration, tokens, turns, output format.
- A ceiling is not a reservation.
--cpuscaps compute; it does not guarantee a minimum. - Detect overruns without relying on the exit code alone, then decide: at most one retry, or stop.
- CPU and memory limits do not cover mounted volumes. Handle disk separately.
- Measure real consumption, then tighten the contract. And measure how stable your results are, not only whether they pass.
The problem
A main agent (deep agent) delegates to sub-agents. Each one runs code, calls tools and consumes tokens. Without per-sub-agent limits, a single defect, such as an infinite loop or an array that grows forever, can take down the whole host. Docker's documentation says it plainly: without limits, when memory runs out, "any process is subject to killing, including Docker and other important applications" (Resource constraints).
The 9 building blocks of a harness, and who provides them
| Block | Question to ask | What breaks without it | Who provides it |
|---|---|---|---|
| Loop | When does the agent stop? | Endless turns | Your harness |
| Tools | What input and output contract? | Wrong tool, opaque errors | You |
| Context | What stays in the window? | Drift | You |
| Permissions | Who authorises what, inside the tool? | Unwanted action | Your tool |
| Execution and quotas | How much memory, CPU and time per sub-agent? | One agent takes down the host | Your infrastructure |
| Coordination | Who owns which resource? | Concurrent writes | You |
| Evals | How do you know it works, every time? | Silent regressions | You |
| Recovery | What does the agent do after an interruption? | Work redone, or declared done halfway | Your harness |
| Cost and logs | Cost per accepted task? | Unexplained bills | You |
This grid is IgnitionAI's, not a standard. This article details execution and quotas: a demo rarely runs long enough to put it to the test.
One execution contract per sub-agent
A sub-agent does not only receive an instruction. It receives a contract, written in a file and validated at startup. Here is the contract of the "code" sub-agent in our example. The values are the example's, deliberately small to trigger overruns: they are not recommendations.
# contracts/code.yaml
name: code
allowedTools: [execute]
memoryMb: 64 # --memory
cpus: 0.5 # --cpus: a ceiling, not a reservation
pidsLimit: 32 # --pids-limit
timeoutMs: 6000 # stopped by the harness
maxTokens: 2048 # sub-agent token budget
maxTurns: 4 # loop turn ceiling
outputSchema:
type: object
required: [language, code]
incidentPolicy:
memory_overflow: retry
timeout: abort
nonzero_exit: retryAn invalid contract does not start, and the error names the field to fix: this is one of the 20 tasks in our eval set. The incident policy lives in the contract, not in the code: the example's "research" sub-agent stops at its first memory overrun.
Docker options: what they guarantee, and what they do not
Sources: Resource constraints and the docker container run reference.
| Option | What it does | What it does not guarantee |
|---|---|---|
--memory | a hard limit: the container cannot exceed the set value | that the agent survives: when memory runs out, the kernel kills processes in the container |
--memory-swap | caps memory plus swap; equal to --memory, no swap | anything without --memory, which it depends on |
--cpus | a compute ceiling, equivalent to a CFS quota | a minimum: it is a ceiling, not a reservation |
--cpu-shares | a relative weight, enforced only when CPU is scarce | precise CPU access |
--cpuset-cpus | use limited to the listed cores | an amount of compute: it picks cores, not the dose |
--pids-limit | limits the number of processes in the container | the page gives neither a default nor the behaviour on overrun |
--storage-opt size=… | the size of the container filesystem, on some drivers; with overlay2, only on xfs mounted with pquota | mounted volumes: not documented, our inference (see below) |
A ceiling is not a reservation. The documentation says --cpus="1.5" guarantees "at most" 1.5 CPUs. If a sub-agent needs a minimum, arrange it elsewhere, for example by reserving cores.
CPU and memory limits do not cover disk. --storage-opt size targets the container filesystem. The --device-write-bps and --device-write-iops options "limit write rate … to a device" (docker container run reference): they bound how fast data is written, not the total volume written. The documentation does not discuss mounted volumes; our inference, to be taken as such: an agent that writes endlessly to a mounted volume can fill the host's disk without touching its memory. For temporary workspace, --tmpfs accepts a size (size=). Otherwise, set quotas at filesystem level.
Docker options have no notion of a task. They bound a container. The "retry or stop" decision comes from your harness.
Detect the overrun, then decide
The harness must know what happened. Our example tells three cases apart, each logged as JSONL:
- Memory overrun: the harness reads the
OOMKilledstate fromdocker inspect. In our demo, the container exits with code 137 andOOMKilled=true. - Time overrun: the harness kills the container when
timeoutMsexpires, then removes it. - Invalid output: the output does not match
outputSchema.
Do not rely on exit code 137 alone. In the same demo, the container killed by the harness for a time overrun also exits with 137, with OOMKilled=false. The two Docker pages cited here do not document this code. Our harness therefore classifies an incident as a memory overrun only when docker inspect confirms it, or when inspection fails, and it then says so in the log.
The policy is simple: at most one retry, with the same limit and a message describing the incident to the model. Without a retry ceiling, a badly sized sub-agent consumes the whole task's budget. If the retry fails, the task stops and the report says why.
In the demo, the memory overrun is followed by a retry that succeeds; the time overrun stops the task, as the contract requires.
Measure before you tighten
Do not set limits at random. Measure real consumption on a task set, then tighten the contract. With Docker, docker stats shows a live stream of container resource usage (reference). On Linux with cgroup v2, memory.max holds a group's hard memory limit and cpu.max its CPU bandwidth limit (Linux kernel documentation). Beware: inside a container, free reports the host's swap, not the container's (Docker).
For concurrency, add up the ceilings of the active sub-agents. If the sum exceeds the host's capacity, queue: ceilings that add up to more than the host do not protect it.
Evals: passing once, passing every time
Check the outcome in the environment, not the transcript. Anthropic recommends starting with "20-50 simple tasks drawn from real failures" (Demystifying evals). The same article separates two measures:
- pass@k: the likelihood of at least one success in k attempts;
- pass^k: the probability that all k attempts succeed.
The first fits when one success is enough. The second fits when consistency matters, which is the case for an agent in production.
Our example runs 20 tasks, most of which trigger an incident: memory overrun, time overrun, denied tool, invalid output, turn ceiling, token budget, invalid contract, recovery after an interruption. With k = 5 attempts per task:
| Measure | Result |
|---|---|
| pass@k | 1.000 |
| pass^k | 0.850 |
All 20 tasks succeed at least once. Three do not succeed all five times, one of them built on purpose to fail at random. pass@k alone would have shown a perfect score. Two full runs, launched separately, produce the same report, apart from the generation date.
For recovery, our example keeps a progress note updated after each saved step. Killed abruptly in the middle of the second step, it resumes without redoing the first; over three interruptions, the final result is identical to a run without interruption.
What we have not solved
- Cost per task is not stable. We wanted to publish the average cost per accepted task. Two identical runs give token totals that differ by a few units, on the memory overrun task. Our hypothesis, unproven: depending on the draw, the Docker daemon does not always report the
OOMKilledstate, and the incident takes another path. So we publish no cost figure. The lesson goes beyond our example: a harness that depends on the state of a shared daemon must check that its measurements are reproducible before comparing them. - Scheduling is not automated. Assigning tasks to sub-agents and setting quotas per task class are still decided by hand.
- The model is fake. The example runs offline, without an API key, with a simulated model. It tests the harness, not the quality of a real model.
Your situation: where to start?
| Your case | First workstream | Can wait |
|---|---|---|
| Internal demo | Contract per sub-agent, logs | Concurrency |
| First agent on a business tool, read-only | Permissions inside the tool, evals | Advanced recovery |
| Sub-agents that run code | Contract, sandbox, overrun detection | Scheduler |
| Agent that writes to external systems | Idempotency keys, quotas, sandbox | Cost optimisation |
Order rule: permissions before autonomy, measurement before tightening.
To scope a project before launching it, see twelve questions to settle before funding an AI project. For agent permissions, see where to place human approvals on business tools.
Are your sub-agents running without an execution contract? Describe your context to IgnitionAI. The first conversation is for spotting what could take down your host, not for presenting a product.
FAQ
Is a memory ceiling enough to protect the host? No. It bounds the container's memory. Disk, mounted volumes and shared compute need separate measures.
Does exit code 137 mean the container ran out of memory?
Not necessarily. In our example, a container killed for a time overrun also exits with 137. Read the OOMKilled state with docker inspect.
Why not retry indefinitely on failure? Because a badly sized sub-agent consumes the whole task's budget. At most one retry, then a stop with a report.
How many sub-agents can run in parallel? As many as the sum of their ceilings fits into the host's capacity. Beyond that, a queue.
Why measure pass^k and not only pass@k? Because an agent in production must succeed every time. In our example, pass@k is 1.000 and pass^k 0.850: the first figure hides three inconsistent tasks.
Methodology and sources
Sources consulted on 11 November 2026:
- Anthropic, Demystifying evals for AI agents (harness definition, pass@k, pass^k, 20-50 tasks)
- Docker, Resource constraints
- Docker, docker container run
- Docker, docker container stats
- Linux kernel, Control Group v2
Measurements: internal test of 11 November 2026 on IgnitionAI's example, version at commit 2ab7714, reproducible internally, in TypeScript, LangGraph JS and Bun, with an offline fake model and node:22-alpine Docker containers. Set of 20 tasks, k = 5, fixed seed. Results of 11 November 2026: 46 of 46 tests passing, two eval runs identical apart from the generation date, recovery demo run three times. The example's code is not public. Deliberately not quoted: any vendor figure without a public method, and our cost per task, which is unstable between two runs.