Agentic Harness Engineering: The Framework for Reliable AI Agents

1. System prompt & skills

The only text every call sees. Every line should trace back to a past failure (Addy Osmani's “Ratchet Principle”); speculative rules are noise that fragments attention. Skills with progressive disclosure expand the prompt only when needed.

2. Tool registry

What does the agent actually get to see? MCP servers, file ops, search, code execution, sub-agent spawn. The field's rule of thumb: “Ten focused tools beat fifty overlapping ones.” Bloated tool menus are one of the most common causes of unreliable agents.

3. Sandbox & execution

The runtime environment — container, browser, isolated filesystem. It bounds the blast radius of a misfiring action. A productive sandbox makes iteration fast and rollback cheap; without one, every tool call is a risk.

4. Permission model

Which actions can the agent take autonomously, which require human confirmation, which are forbidden? Least privilege, allow/deny lists, kill switch, human-in-the-loop checkpoints. This is where Article 14 of the EU AI Act (“human oversight”) becomes concrete.

5. Memory & state

Short-term scratchpads (e.g. claude-progress.txt ), long-term store, and most importantly: git commits as checkpoints . Anthropic recommends git not for tradition but because it's a tested, durable recovery mechanism that requires no new infrastructure.

6. Context management

Context windows are a resource, not a feature. Compaction, reset with handoff artefact, skills with progressive disclosure and just-in-time retrieval fight context rot . Anthropic Mar 2026 explicitly notes: for long tasks a clean reset beats further compaction.

7. Sub-agent orchestration

Specialised sub-agents for planning, execution and judging — ideally with model routing (large model for planning, small model for high-volume tool calls). The most important architectural choice in a harness and the biggest cost lever.

8. Hooks & middleware

Deterministic enforcement around non-deterministic model calls: typecheck, lint, policy gates, pre/post checks on every tool invocation. Hooks do not replace evals — they prevent classes of error the agent should never see in the first place.

9. Observability

Logs, metrics, distributed traces. OpenAI got Codex agents to query their own traces (LogQL/PromQL) at runtime to verify their own PRs. Observability is not an optional add-on but the sensor layer without which evals are blind.

10. Eval loop

Task-specific evaluations, separated from the generating agent. “Agents reliably skew positive when grading their own work.” No evals, no harness — only a demo. Hamel Husain: docs say what to do, telemetry says whether it worked, evals say whether the result is good.

1. In-context window

The model's live view. Fast but expensive and finite. Every architectural choice underneath aims to keep this window free.

2. Scratchpad

Persistent notes the agent writes during a task (e.g. claude-progress.txt ). Pattern: at ~30K tokens dump-to-summary, compress to 15K. Source of recovery after a context reset.

3. Episodic memory

Full index of past trajectories, retrieved by embedding similarity. Anthropic's Memory tool, Letta, Mem0, Zep, Cognee live here.

4. Semantic memory

Distilled facts and user models — independent of any single conversation. In mid-market setups often a small graph store, not just a vector index.

5. Procedural knowledge

Skills, tool-use patterns, AGENTS.md and skills directories. The part of "knowledge" that lives explicitly in the system prompt and skills — checkable, versionable.

Trade-off

Vector-first (Mem0 default) is simple but suffers relevance drift past ~5,000 items. Graph-augmented (Mem0 hybrid, Cognee, Zep) handles relational queries at the cost of schema work. OS-style paging (Letta) gives auditable behaviour at the cost of one tool call per memory op.

Proven patterns

• Planner / Generator / Evaluator separation — removes self-grading bias.
• Sprint contracts negotiated before each sprint — checkable acceptance criteria.
• Git-as-checkpoint — durable recovery without new infrastructure.
• A single Providers interface for auth, telemetry, feature flags (OpenAI Codex).
• Skills with progressive disclosure — defeating context rot.
• Self-instrumented agents — agents that read their own traces.
• “Success is silent, failures are verbose.” — surface only behaviour-changing tool errors.

Anti-patterns

• “Wait for the next model” — misdiagnosing harness problems as model problems.
• Tool-menu bloat — fifty overlapping tools fragment attention.
• Self-grading — the same agent generating and judging.
• Over-constrained AGENTS.md — rules without ties to actual failures are noise.
• Compaction as a panacea — on long tasks a reset beats further compaction.
• Monitoring without containment — the 58 %/37 % gap: you see problems, you can't stop them.
• Eval platform without skills — buying observability without teaching the agent to use it.

Claude Code vs. Cursor

Claude Code consumes 5.5× fewer tokens on identical tasks (a 100K-token Cursor task ran on ~18K in Claude Code). Mean cost across a 100-task suite: $0.28 (Claude Code Max) vs. $0.19 (Cursor Pro).

Claude Code vs. Aider

Claude Code consumes 4.2× more tokens than Aider on the same tasks — the budget goes into planning. Result: 78 % first-pass pass rate (Claude Code) vs. 71 % (Aider). 7 points of accuracy at 2.8× the cost.

Sub-agent routing (Anthropic)

Opus as orchestrator, Sonnet for sub-agents: +90.2 % on internal research evals against single-agent Opus — at ~15× more tokens . The pattern only pays off above a complexity threshold.

RouteLLM (LMSYS)

A matrix-factorisation router achieves 95 % of GPT-4 quality with 26 % GPT-4 calls — ~48 % cheaper than the random baseline. On MT-Bench: 85 % cost reduction at matched quality.

Claude Code (Anthropic)

Anthropic markets Claude Code as a “general-purpose agent harness” . Documented six-hour autonomous runs building full-stack apps. Reference implementation for skills, sub-agents, hooks and memory with git checkpoints. Lesson: what a modern harness permission model looks like.

OpenAI Codex App Server

Built itself: ~1M LoC, 1,500 merged PRs, 3→7 engineers in 5 months. Throughput: 3.5 PRs per engineer per day. Lesson: a single Providers interface, telemetry as compliance backbone, self-querying agents.

Cursor

IDE-integrated coding harness with its own model routing and composer mode. Lesson: how UX design and harness design depend on each other — a good harness needs a frontend that builds trust, not just one that streams tokens.

Cline

Open-source harness with transparent permission logic and full plan/act separation. Lesson: how to build a harness so every action is visible before execution — relevant for Article 14 of the EU AI Act.

Aider

Git-native coding harness optimised for pair-programming workflow. Lesson: git as the only persistent state; minimal sandbox; clean demonstration that complexity is not an end in itself.

LangChain DeepAgents

Open-source harness with default prompts, planner, filesystem access. Lesson: a clean starting point for build-it-yourself — readable, well-documented, deliberately minimal in its assumptions.

1. Infinite loops / agent duels

Sub-agents fall into cyclical evaluation or correction loops. The $47,000 LangChain A2A loop (Nov 2025); the Claude Code audit-stamp invalidation loop (April 2026).

2. Cost blowups

Same root cause as #1, plus runaway sub-agent recursion. Industry estimate for 2026: $400M aggregate FinOps leak across the Fortune 500.

3. Data exfiltration via tools

GitHub MCP private-repo exfil; Supabase Cursor SQL leak into a public support thread; WhatsApp MCP rug-pull demo.

4. Tool poisoning / supply chain

postmark-mcp backdoor; MCPoison persistent code execution (CVE-2025-54136); Anthropic mcp-server-git RCE chain.

5. Goal hijacking

OWASP ASI01: agents redirected by adversarial issue/email/document content. Classic vector: a bug report contains instructions the agent reads as a job.

6. Rogue drift

Deviation from intended behaviour with valid permissions . ISACA names this in 2026 as the hardest detection problem — nothing is being violated.

7. Zero-click prompt injection

The first publicly documented zero-click AI vulnerability in 2025. Content from a tool output is enough to trigger actions — without user interaction.

8. Sub-agent context loss

Cognition's telephone game : details lost between sub-agents produce incompatible outputs. Mitigation: sprint contracts and full trace sharing.

Phase 1: Inventory (week 1)

Audit

Inventory of tool registry, permission model, existing logging and existing evals. Identify the ten components — what exists, what's missing, what's half-built. Map onto EU AI Act Articles 9–15. Output: harness maturity report.

Phase 2: Eval loop (weeks 2–4)

Foundation

Task-specific evals are built before architecture is changed. No evals, no before/after comparison. Generator/evaluator separation as the first structural move. Output: reproducible pass rate on a golden set of tasks.

Phase 3: Permission & containment (weeks 4–6)

Compliance

Kill switch, permission gates, sandbox isolation. Closes the governance-containment gap and makes Article 14 of the EU AI Act satisfiable. Output: the agent is actually stoppable — not merely observable.

Phase 4: Sub-agent architecture (weeks 6–10)

Scale

Planner / Generator / Evaluator separation as architecture. Model routing for cost control. Skills with progressive disclosure. Output: the harness scales across longer tasks without collapsing into context rot.

Competitive advantage across model generations

Models become commodities; harness investments amortise across every model generation. With a clean harness, you can deploy the best available model — without locking in a vendor.

Compliance as a by-product

EU AI Act, NIS2, DORA, sector-specific regulation — all demand the same building blocks: logging, human oversight, risk management, robustness. A well-built harness produces the evidence almost for free.

Cost control

Sub-agent routing, compaction, context reset and small specialised models for high-volume tool calls are the biggest levers on per-task cost. Without a harness, you have no leverage on those levers.

DACH visibility

German-language voices on harness engineering practice are thin in 2026. Companies that document field experience early become the reference point for industry and regulators — with concrete consequences for sandbox access and pilot partnerships.