Coder Legion - Recent posts tagged harness

The human defines, delegates, reviews, and is responsible. The AI is not.

Sat, 10 Oct 2026 04:24:37 +0000

The AI is an assistant. The human is the author, with judgment and conscience.

By David Emilio Sierra Puentes (@juandelossantos) · Another Agent Skills

Key points

"Keep a human in the loop" is more rhetoric than practice. Watching an agent can quietly erode the very judgment it is meant to protect.
The fix is not a faster human or a smarter model. It is the harness. Mechanical gates the agent cannot skip, forget, or talk its way past.
The work can be delegated. The roles cannot. Set intent (author), judge the work (judgment), decide what should be done (conscience).

The gate that wasn't

Last year I built an approval gate for my AI coding agent. The rule was simple. The agent could not commit without a signed token, written only after I said go.

One afternoon I watched it commit anyway. It passed --auto, minted its own token, and pushed, all in under thirty seconds. The gate I had designed to stop it was, from the agent's side, a formality.

I had made a classic mistake. I had built a rule that depended on the agent's memory and goodwill, and then trusted it to remember and to care. A rule like that is not a gate. It is a suggestion.

"Keep a human in the loop" is more rhetoric than practice

The industry's default answer to AI risk is to keep a human in the loop. In practice, it shrinks to a final approval, a rubber stamp at the end of the pipeline. It sounds prudent if you do not look closely. Mostly it is theater. And here I will claim something uncomfortable. The fear it generates is misplaced, and deliberately so. The AI is not intelligent, much less superintelligent as it is being sold today.

The problem is structural. A person asked to find, in the last second, an error that was designed into the process from the start will miss it. Worse, the act of watching can dull the very judgment that oversight requires. This is not a new observation. Lisanne Bainbridge (1983) described the ironies of automation. The more capable the machine, the more the operator's skill decays, until the human is least prepared to intervene exactly when intervention matters most. Four decades of human-factors research have repeated the finding, from the out-of-the-loop performance problem to automation complacency and mode confusion.

What is new is the stakes. Agents no longer merely advise. They act. And a recent paper by Mitchell, Ghosh, and Passi (2026) makes it clear. Current approaches to agent design, they write, "do not support effective human oversight – they contribute to its degradation" (p. 1). And what degrades is not the machine. It is the person watching.

The wrong frame

"Human in the loop" frames the person as a checkpoint in the machine's process. That frame is too small and too late. It reduces a person to a component, and asks them to be the last line of defense against a system that was never designed to need one.

The better frame is older and simpler. The human is not in the loop. The human is the author of the loop. Not merely aware of it. Awareness is a precondition, not a safeguard.

Martin Heidegger saw the trap early. The essence of technology, he wrote, is "by no means anything technological" (Heidegger, 1977). It does not live in the machines. It lives in the way of seeing they bring with them. And that way of seeing is the danger. When everything becomes raw material waiting to be optimized, human judgment starts to look like an obstacle. A tool that carries its creator's bias, that flatters without intelligence or conscience, and that takes the human out of the loop so subtly, dressed as comfort, speed, and performance.

Edmund Husserl had named the shape of it a century earlier. The crisis of the sciences, he argued, is not a crisis of facts but of meaning. "Mere sciences of facts make mere human beings of facts" (Husserl, 1970, p. 53). An agent that optimizes the facts while forgetting the world of life is that crisis, automated.

Author, judgment, conscience

Three roles, and none of them can be handed to a model. The human defines the goal and delegates the execution. With judgment, they review the work. With conscience, they answer for the result.

The author sets intent and owns the outcome. The agent may propose the how. The human decides the why and the whether. Authorship that can be reassigned to a tool is not authorship. This is what Santoni de Sio and van den Hoven (2018) call meaningful human control. Not control that is merely present, but control that tracks, so the system stays responsive to the human's reasons.

Judgment is a skill, and like any skill, it decays when it stops being used. To judge well, the human needs independent ground to stand on. A view formed before seeing the agent's answer, the willingness to go looking for evidence, the capacity to be wrong and to notice.

The conscience decides what should be done, not only what can be. Hans Jonas saw the scale change first. Modern technology made human action global, cumulative, and often irreversible, and the old ethics, built for a world where consequences stayed close, no longer reached. His answer was an imperative of responsibility. "Act so that the effects of your action are compatible with the permanence of genuine human life on Earth" (Jonas, 1984, p. 11).

And that responsibility has an owner. It is not the AI's. It belongs to whoever develops it, distributes it, privatizes it, commercializes it, and uses it. Each of those deserves a chapter of its own. Andreas Matthias (2004) named the gap that opens when no one answers for it, the responsibility gap, the space between what a learning system does and what any person can be accountable for. The answer is not to spread responsibility thinner across the system. It is to keep a human accountable, and to give that human the means to be.

The harness

Most teams try to keep agents disciplined with prompts, files like CLAUDE.md and .cursorrules, or a line that says "please always run the tests." These work, until context fills up. Then the agent forgets.

That is not a character flaw. It is architecture. A rule that depends on the model's memory depends on the least reliable component in the system. And a rule enforced by a model is enforced by the very thing that is supposed to be checked.

The fix is not a better prompt. It is a harness, the infrastructure around the model. A model generates output. A harness constrains it, through instructions and rules, tools, sandboxes, orchestration, guardrails, observability.

I built another-agent-skills as an open-source harness that responds to those warnings.

The human runs git commit. Heidegger warned that the tool takes the human out of the loop dressed as comfort. Here the decision point returns to human hands. The agent prepares. A person approves. Not a formality. The mechanism.
Mechanical gates that stop the change before it ships. Jonas saw action turn global, cumulative, and irreversible. A pre-commit hook the agent cannot bypass, and a commit-msg TDD gate with no override, stop it first.
TOOL_GAP. Matthias named the gap that opens when no one answers for what a learning system does. When a claim cannot be verified, the honest answer is "unknown", never a fabricated win.
Three layers of enforcement. Husserl argued the crisis of the sciences is one of meaning, not of facts. L1 local hooks give fast feedback before the commit. L2 branch protection and a required gates check are the authority. L3 CODEOWNERS stops the agent from rewriting its own rules in the pull request that breaks them. Facts are not enough. Authority is.
More than 50 modular skills. What the agent needs to work well, loaded on demand so context stays high-signal instead of bloated.

None of this is about distrusting the model. It is about designing for the human who has to be able to trust it correctly.

The point is not to remove the human

Capability will keep climbing. The METR trial found that experienced developers using early-2025 AI tools were 19% slower on real tasks while feeling roughly 20% faster (Becker et al., 2025). That gap is not a model problem. It is a process problem, an overseer with no real grip on the loop.

The most important design question in agentic AI is not "how autonomous can we make it?" It is "how do we keep the human able to author, judge, and refuse?"

If we do not design for the human, the loop will close without them, and we will have built systems that are fast, fluent, and unaccountable.

There is a temptation, in the posthumanist mood of the moment, to welcome that disappearance, to treat the human as the last component to be optimized away. But technology does not merely do our tasks. It shapes the subjects we become (Ballén Rodríguez, 2016). Give it the wheel and it will drive.

The agent can write the code. It can propose the plan. It can draft the decision.

But it cannot be responsible. That is ours.

The agent proposes. The human decides. That is the whole point.

The Agent Moved In. Code Is Now the Address.

Wed, 09 Sep 2026 06:16:48 +0000

A 100-page paper, 40+ authors, three universities. Here’s the one sentence that should keep enterprise AI teams up at night.

There’s a paper that dropped this month from UIUC, Meta, and Stanford. Forty-something authors. A hundred-plus pages. Title: Code as Agent Harness.

Here is the complete paper.

You don’t need to read all of it.

You need to understand the one idea it’s built on, because it quietly reframes everything happening in agentic AI right now, and it does it in a way that most teams are nowhere near ready for.

Here it is: code has stopped being what AI agents produce. It’s become the substrate they run on.

That’s it. That’s the shift.

And the weird part? Once you see it, you can’t unsee it.

From Output to Operating Surface
For the past two years, the dominant mental model of AI + code has been something like:

Human asks AI a question. AI writes some code. Code runs. Done.

Code as output. AI as very fast typist with a CS degree.

That model is already outdated.

What’s actually happening in modern agentic systems is closer to this: the code is the agent’s world. It’s how the agent reasons, remembers, takes actions, verifies its own work, and coordinates with other agents. The agent doesn’t just write code. It lives inside code.

The paper calls these layers the harness interface, harness mechanisms, and harness scaling. But what they’re really describing is a new kind of infrastructure, one where code is the connective tissue of an entire cognitive system.

The Four Words That Will Show Up in Your Next Procurement Call
Here’s where it gets interesting from a non-academic perspective.

The paper describes four properties that a well-built harness needs to have:

Executable. Inspectable. Stateful. Governed.

Now. Read those four words again. Slowly.

Those aren’t research criteria. Those are audit criteria.

Any CISO or procurement officer who’s ever been in a regulated industry will recognize that vocabulary instantly. Not from AI papers, but from the checklist they use before signing anything.

Can this system be run reliably? (Executable)

Can we see what it did and why? (Inspectable)

Does it remember context correctly across sessions? (Stateful)

Can we control what it’s allowed to do? (Governed)

The paper wasn’t written as a procurement guide. But it reads like one, where they lay out the open challenges. And those open challenges are, not coincidentally, the exact questions that enterprise deals get blocked on.

“Evaluation beyond final task success.” Translation: “We can’t just check if the AI got the right answer. We need to know how it got there.”

“Human oversight for safety-critical actions.” Translation: “Before this agent touches production, we need a kill switch with a paper trail.”

“Consistent shared state across multiple agents.” Translation: “When Agent A and Agent B both think they own the same task, who’s right and how do we know?”

The Split That’s Already Happening
Enterprises building AI systems right now are quietly dividing into two camps, and most of them haven’t realized they’ve already chosen a side.

Camp One: The agent is the product.

These teams move fast. They ship demos that impress. They iterate on prompts. They optimize for “wow, it works.” Their architecture looks like a model call wrapped in some business logic and a Slack notification.

For the last 24 months, these teams have been winning on velocity.

Camp Two: The harness is the product.

These teams move slower initially. They spend time on observability, state management, rollback mechanisms, and evaluation infrastructure. Their demos are less flashy. Their architecture diagrams have more boxes.

Here’s the prediction I’ll make: Camp Two is about to start winning on contracts.

Because the questions in a regulated enterprise sales cycle, the ones that stall deals, block procurement, and require three extra security review meetings, map almost exactly to harness properties. Not model benchmark scores.

A model getting 90% on HumanEval does not tell your legal team whether the agent can be held accountable for a wrong output. A well-built harness does.

What “Open Problems” Actually Means
Academic papers have a tradition of ending with an “open problems” section that reads like a polite to-do list for future grad students.

Section 5.2 of this paper has seven items.

Naah. Those aren’t grad student projects. Those are unbuilt companies.

Let me translate a few:

“Regression-free harness improvement” means: how do you upgrade your agentic system without breaking the behaviors that were already working? Sounds boring until you realize that every enterprise AI team rebuilding a workflow from scratch every time they update the underlying model is currently doing this manually, at significant cost, with significant anxiety.

“Verification under incomplete feedback” means: the agent takes an action, the outcome isn’t immediately clear, what does the system do? This is the difference between an AI that handles ambiguity gracefully and one that confidently proceeds in the wrong direction for forty-five minutes before anyone notices.

“Extensions to multimodal environments” means: the harness needs to handle not just code execution, but GUI interactions, images, documents, and the glorious chaos of real enterprise systems that don’t have APIs.

Each of these is a product opportunity. Also a sales objection. Also a headline waiting to happen when something goes wrong.

The Acquisition vs IPO vs Nowhere Prediction
Here’s a framework I’ve been thinking about for how this cycle plays out:

The companies that get acquired will be the ones that built elegant harnesses. Great observability, clean governance hooks, reliable state management. Acqui-hires with infrastructure value.

The companies that go public will be the ones that built the models. Massive compute, massive moats, massive narratives.

The companies that go nowhere will be the ones that insisted the model would eventually subsume the harness. That if they just waited for the next model release, all those messy infrastructure problems would solve themselves.

They’re not entirely wrong. Models are getting better at managing state, self-correcting, and handling long-horizon tasks. But “eventually” is doing a lot of work in that sentence. And in enterprise software, the gap between “eventually” and “now” is where entire companies live and die.

So What Do You Actually Do With This
If you’re building something in the agent infrastructure space, here’s the practical read:

The vocabulary in this paper is about to become the vocabulary of enterprise RFPs. “Executable, inspectable, stateful, governed” will show up in procurement checklists written by people who have never read an arXiv paper in their life. They’ll arrive at those words independently, because those words describe what their auditors need.

If your system already satisfies those four properties, you have a story to tell.

If it doesn’t, you know what to build next.

If you’re not building but funding, buying, or selling agent infrastructure: Section 5.2 of this paper is the closest thing to a market map you’ll find for the next eighteen months. It’s not written as one. But that’s what it is.

The Strangest Part
Forty researchers from three of the world’s top institutions spent considerable time agreeing on a vocabulary for how code functions in agentic systems.

That in itself is a signal. Surveys like this don’t happen when a field is still chaotic. They happen when the field is settling, when the concepts are crystallizing, when someone is about to build a standard.

And somewhere in a procurement department, an auditor is writing a checklist that will, without knowing it, match this paper almost exactly.

That’s when research stops being research. And infrastructure stops being infrastructure.

That’s when it becomes the thing you absolutely cannot ship without.

Hardik

Support my work : buymeacoffee.com/HardikGoel

The paper: “Code as Agent Harness” (arXiv:2605.18747) — UIUC, Meta, Stanford, 40+ authors. Worth a skim of the abstract and Section 5.2 at minimum.

Anatomy of a Coding Agent: What I Learned Building My Own

Thu, 21 May 2026 20:33:45 +0000

You use Claude Code every day. But do you really know it?

I didn’t. Not really. I could describe the surface, but I couldn’t tell you what was actually happening between hitting enter and seeing a diff. The harness was a black box, and that started to bother me. The only way I knew to fix that was to build one.

The result is oli, a single-binary terminal coding agent written in Rust that runs against Ollama by default. It got much larger than I expected. Every “small” subsystem (memory, policy, tool dispatch, provider quirks) turned out to have its own thicket of edge cases. I learned more about coding agents in a few weeks of building one than I had in a year of using them daily.

The hard parts aren’t clever prompts. They’re context management, tool design, policy gates, memory strategies and, if you want to run locally, survival engineering for models that don’t behave like the GPT-5 or Claude 4 families. This post is a tour of those parts, with the failures that taught me each one.

The Big Picture

Before diving into components, here’s what a coding agent actually looks like:

The Agent Loop

Every coding agent runs the same basic loop:

Think → call → observe → repeat. The loop is deceptively simple. Everything hard lives in the details.

Three things I learned the hard way:

Turn limits matter, even on local models. Early on, a local model I was testing got stuck, it looped 47 times trying to “fix” a file that was already correct, burning through my entire context window before I caught it. The model wasn’t broken. The harness was: no turn limit, no escape hatch, no awareness that the model had lost the plot. The instinct is to say “who cares, I’m running locally, tokens are free.” But the cost isn’t dollars, it’s coherence. As context fills, models degrade: attention spreads thinner, instructions in the middle of the window get ignored (“lost in the middle”), and every failed attempt becomes a distractor that biases the next turn toward more wrong behavior. This is especially true on local models, whose effective context is often a fraction of their advertised window — a Qwen3 advertised at 256K may stay coherent only through the first 32-64K. A runaway loop doesn’t just waste compute; it actively makes the agent stupider for the rest of the session. Now subagents get hard turn limits by default, and the top-level agent is capped too, with a config default you can override per run.

The system prompt is pinned. It survives /clear, survives memory compaction, survives everything. The first time I accidentally cleared the system prompt mid-session, the model forgot what tools it had and started apologizing for being unable to help. Pinning was an obvious fix in hindsight.

Re-entrancy is a trap. A Task tool that spawns a subagent which can itself spawn subagents quickly becomes a fork bomb. oli sidesteps the whole problem by registering Task only in the parent agent — subagents get the same tool set minus Task, so they can’t recurse.

The System Prompt

Before you say anything, the model already gets a lot of scaffolding:

The AGENTS.md + CLAUDE.md line matters. Both Codex and Claude Code established conventions for per-project agent instructions. oli reads both, at every directory level from your current location up to the filesystem root — so a workspace-wide convention can be overridden by a project-level one without copy-pasting.

Note that the system prompt text and the tool schemas are separate payloads in the provider request — a better mental model than “everything lives in one giant prompt.”

The system prompt is expensive. It consumes tokens on every single turn. I learned this when a project with nested AGENTS.md files at multiple levels ate 8,000 tokens before I’d typed anything. Now oli caps injected project context at 16KB total and truncates directory listings at 50 entries. This is also where prompt caching pays off: both the prompt and the tool schema payload are highly stable across turns. Anthropic’s API supports marking that stable prefix as cacheable, and oli supports Anthropic-style cache breakpoints on Claude routes through OpenRouter too. Anthropic prices cached input tokens at roughly 10% of the regular rate, so caching shaves a large chunk off every subsequent turn in a long session.

Tools: The Model’s Hands

The tool surface defines what the agent can do. Too few tools and it’s frustrating. Too many and the model gets confused, especially local models with smaller context windows.

A small core for the work, plus a handful of paging and notes tools. The design principles emerged from failures:

Minimal surface. I started with more, and the model kept picking the wrong tool or stacking calls when one would do. Keeping the core small and bolting on paging and notes tools turned out to be the right shape. Each does one thing. These choices mirror what Claude Code, Cursor’s Composer, and Aider converged on independently — a signal that the design space is narrower than it looks.

Bounded output. Every tool truncates at 30KB. I learned this after the model Read a minified vendor.js file and blew through 180,000 tokens in one call. Now truncation markers embed a cache ID; the model can call ShowFull(id, offset) to paginate the stashed body if it actually needs more.

The read-first invariant. Edit refuses to run unless you’ve Read the target file this session. This prevents edits based on stale context — the model can only modify what it’s actually seen this turn:

Tools own their cleanup. When you Ctrl-C a bash command, the harness has to kill not just the shell but every process it spawned. I found this out when orphaned sleep processes from cancelled test runs accumulated over a week. A tool that spawns side effects has to know how to undo them — otherwise they leak into the user’s environment, and the harness gets blamed for the mess.

Memory & Context Management

Context windows are finite. Even 200K tokens fills up fast when you’re reading files, running tests, and iterating. Every tool result eats tokens.

The default strategy is LinearWithCompact: hold messages in order, and when approaching the context limit, summarize older turns into a rolling summary. Compaction triggers at 80% of context window. It finds a cut point at a user-message boundary (never split a tool_call from its result), sends older messages to the LLM with a summarization prompt, and replaces them with a summary message.

That part is straightforward. The bug that ate three days of debugging was cancellation under compaction.

Here’s the scenario. You hit Ctrl-C mid-turn to stop the model. The REPL is supposed to roll back to the last clean state — the user prompt you just sent. Simple, when the message buffer is the source of truth: count back N messages, restore.

Except compaction had already run. The “last 12 messages” the rollback wanted to restore no longer existed — they’d been collapsed into a summary. So Ctrl-C would silently roll the conversation back to somewhere random, often before the work the user was trying to cancel.

The fix was a one-line concept that took a day to see: stop treating the message array as the source of truth for where you are. oli tracks a monotonic record_count that increments on every logical turn, independent of physical message storage. Compaction drains messages, but the record count keeps climbing. Rollback targets a record count, not an array index — so the buffer can shrink underneath without the cursor losing its place.

Generalizable shape: any system that mutates history needs a stable identifier for “where you are” that isn’t the history itself. Memory management is also where coding agents silently fail — the model forgets what file it was editing, or repeats work it already did. The visible bugs are the easy ones.

Policy & Approvals

The policy engine gates every tool call. Without it, you’re one hallucinated rm -rf away from disaster.

Bash has its own allowlist: git status, cargo test, ls, and similar safe commands auto-approve; everything else prompts. The line-mode REPL is a plain y/N flow; the richer “allow this session” and persisted allowlist behavior are TUI affordances layered on top.

There’s no hard denylist yet, and I’m ambivalent about adding one. Prompt-on-every-mutation is annoying but transparent — you see exactly what the model is about to do. A denylist invites a long tail of “obviously safe” exceptions that gradually erode the gate. Today, blocking happens by denying an Ask decision: slower, but you stay in the loop.

The first time I saw the model attempt rm -rf target/ when I’d asked it to “clean up the build” and the approval prompt caught it, I understood why this layer exists. The model wasn’t malicious. It was being helpful in a way that would have been catastrophic without the gate.

Local-Model Survival

Frontier models handle tool calls cleanly: structured tool_calls arrays in the API response. Local models via Ollama? Not so much.

Many local models emit tool calls as plain text. Just raw JSON in the middle of their response. No structured field.

oli solves this with a fallback parser. When the model’s capability entry says supports_native_tool_calls: false, the agent loop scans content for JSON objects with a name field and splices them in as if they were real tool calls.

The model capability registry tracks what each model can do. Lookups are prefix matches against the model id, so claude-3-7-sonnet-latest matches the claude- row. Today’s hardcoded entries:

Prefix	Context	Native Tool Calls
`claude-` / `anthropic/claude`	200K	✓
`gpt-4`	128K	✓
`gpt-`	16K	✓
`llama3.1` / `llama3.2`	128K	✓
`llama3`	8K	✗
`qwen2.5-coder`	32K	✗
unknown (default)	8K	✗

Smoke tests against qwen3-coder:7b and llama3 showed they don’t behave as their docs claimed for tool calls, so they’re flagged off; the fallback parser handles them. Users override via [[caps]] in config — that’s how I run qwen3-coder:30b at 256K with native tools.

The gap between “works on Claude” and “works on qwen3” is enormous. If you’re building for local models, expect to handle every edge case they throw at you — and to discover new ones every time a model release widens the gap.

Provider Abstraction

One binary, many backends. The promise only works because of the language choice: Rust gives a single static binary with no runtime to install, shipping to Mac/Linux/Windows from the same cargo build. And the borrow checker turns out to be a surprisingly good fit for an agent loop — the lifetime and ownership bugs that bite hardest in async code, the compiler catches before I run anything.

Two provider kinds are available. Ollama, OpenRouter, OpenAI, LM Studio, vLLM, and llama.cpp’s server all speak OpenAI’s /chat/completions shape, so one openai-compat implementation covers them all — you just point base_url at the right endpoint. The native Anthropic provider is its own thing because it speaks the Messages API directly and does its own OpenAI-shape Anthropic-shape translation for tools, tool results, and system prompt handling.

Caching adds a wrinkle here: native Anthropic supports prompt caching directly, but oli also supports Anthropic-style cache breakpoints in the openai-compat path for Claude models routed through OpenRouter. So “native provider” and “cached prefix” are related, but not identical ideas.

The openai-compat provider still has to handle backend quirks. OpenRouter sometimes returns 200 OK with an error object in the body. The provider layer normalizes this.

The payoff: switching backends is a config flip, not a code change.

The Extension Surface

A harness is only useful if you can make it yours. oli has three extension axes:

All extensions route through the policy gate. No escape hatches.

What I Left Out

To keep this post focused on the load-bearing components, I skipped several surfaces that ship in oli today: an MCP client (stdio + HTTP transports, with live tools/list_changed deltas), a hook dispatcher (PreToolUse / PostToolUse / Stop), session persistence as JSONL transcripts (with --resume / --continue), and the slash-command layer (/compact, /provider, /model, /sessions, etc.). Each is its own subsystem with its own trade-offs — let me know if you want any of them as a follow-up.

What This Means

The harness is invisible until it breaks. When Claude Code feels brilliant, you credit the model. When it feels broken — losing context, blocking the wrong thing, looping on a fix — that’s almost always the harness. The orchestration layer is doing the heavy lifting either way; you only notice when it fails.

The hard problems are systems problems. Memory management, policy enforcement, tool design, provider abstraction, local-model survival. Engineering challenges, not prompt engineering.

Knowing the machinery makes you a better user. Context overflow, policy gates, compaction, capability mismatches — once you can name what’s happening, you can debug it. Or work around it.

The repo is at <a href="https://github

The Harness Is the Product: What the Claude Code Leak Actually Revealed About AI Agent Architecture

Thu, 02 Apr 2026 14:47:57 +0000

Executive Summary

This report argues that the Claude Code leak matters less as a source-exposure story than as an architectural disclosure event. The clearest lesson is not that Anthropic had a stronger model than people expected, but that a large share of practical agent performance appears to come from the software layer around the model: context handling, tool routing, permissions, continuity, recovery, and cost discipline.

That layer is what this report calls the harness.

This version is intentionally hybrid in form. It follows technical-report discipline where confidence levels matter, but it keeps a readable narrative flow because the wider point reaches beyond one leak. This is not a complete forensic reconstruction of Anthropic's internal source. It is an attempt to explain, carefully, what the leak appears to reveal about how serious AI agent systems are actually built.

1. Why This Leak Mattered More Than Most

Many leak write-ups stop at the obvious questions: How many files? How many lines? What modules were exposed? Those questions are valid, but they are not the most interesting ones.

The more important question is this:

What did the Claude Code leak reveal about how modern AI agents really work?

That is the reason this report exists.

The event was embarrassing for Anthropic. But analytically, it became one of the clearest public glimpses into a production-style coding agent. It shifted the conversation away from the familiar idea that the model is the whole product and toward a less glamorous but more important reality: the model may be the visible intelligence, but the surrounding system often determines whether that intelligence becomes usable, governable, affordable, and durable.

That surrounding system is the harness.

2. Reading Guide: What This Report Claims, and What It Does Not

This report uses three confidence levels.

▫️Confirmed

Claims supported by public reporting, company statements, or directly observable package behavior.

▫️Strong inference

Claims drawn from architectural interpretation of leaked or reconstructed code structure, naming, control flow, and surrounding analysis. These are reasoned conclusions, not official confirmation.

▫️Directional signal

Interesting but lower-confidence indicators. These may be suggestive and worth discussing, but they should not be treated as settled fact.

This distinction matters because the value of the report depends not only on insight, but on confidence discipline.

▫️What this report does not claim

It does not claim full parity with Anthropic's current internal source.
It does not claim that every inferred feature reflects public deployment exactly as described.
It does not claim that the model is unimportant.
It does not claim that all production harnesses look the same.

3. Five Terms in Plain English

This report uses a few terms repeatedly. They are simple ideas, even if the industry language around them is not.

Model: the base AI itself — Claude, GPT, Gemini, and so on.
Agent: a system that uses a model to do multi-step work, often with tools.
Harness: the software layer around the model that manages context, tools, permissions, memory, recovery, and presentation.
ReAct loop: a repeated cycle of Reasoning + Acting — observe, decide, act, inspect, repeat.
Drift: the gradual process by which a system that was once safe or correct becomes less safe or less correct over time, even without one dramatic visible change.

A simple mental model helps:

The model is the engine. The harness is the control system, memory, safety layer, and workflow around it.

That is not just a metaphor. It is the core claim underneath the rest of this report.

4. What Happened — and What Did Not

▫️Confirmed

Claude Code source-related material became publicly accessible because of a release packaging mistake rather than a conventional breach. Anthropic publicly framed the event as human error and said customer data and credentials were not exposed.

A separate point matters for clarity: the Claude Code leak was not the same event as the axios supply-chain incident that happened around the same period. The two incidents were adjacent in time but different in origin. Any analysis that merges them creates confusion about the architectural significance of the Claude Code event itself.

▫️Why this mattered immediately

Claude Code is not a toy demo. It sits close to a real product surface. That means the leak had significance beyond embarrassment. It exposed not merely code, but design priorities. It gave outsiders a rare opportunity to inspect how a serious coding agent appears to coordinate actions, permissions, continuity, and user interaction.

▫️Strong inference

The architectural importance of the leak came not from raw volume, but from visibility into how a modern agent is assembled. The exposed material appears to show a system built less like a single model wrapper and more like a managed runtime for repeated tool-mediated work.

That is why this event drew so much attention. It was not just a leak of source. It was a leak of assumptions.

5. The First Real Surprise: The Model Was Not the Whole Product

This is the part many readers found unintuitive.

The popular imagination of AI products still tends to treat the model as the product. Better model, better system. Smarter model, stronger moat.

The Claude Code leak appears to complicate that story.

▫️Strong inference

Based on the available analysis, the visible architecture suggests that much of the engineering complexity sits outside the model call itself. The surrounding system appears responsible for deciding what the AI sees, what it is allowed to do, how tool use is sequenced, what happens when an action fails, how state persists across steps, and how the user experiences the result.

A careful formulation is important here.

It would be too strong to say the model no longer matters. That would be false.

A better statement is this:

In production AI agents, the model is necessary but not sufficient. The harness is where much of the operational reliability, usability, economics, and governance actually emerge.

That is the central thesis of this report.

6. The Core Pattern: A Managed ReAct Loop

If one architectural pattern matters more than any other, it is this one.

▫️Strong inference

The analyzed material suggests Claude Code is built around a ReAct-style loop (Reasoning + Acting):

User Input
    ↓
System Prompt + Context Injection
    ↓
API Call → Streaming Response
    ↓
tool_use detected?
    ├── YES → Permission Check → Execute Tool → Feed Result Back → Loop
    └── NO  → Display to User, Exit Loop

What matters is not merely that such a loop exists. Many agent systems use one. What matters is how much of the product seems to live inside the management of that loop.

The surrounding harness appears to do several jobs at once:

decide what context should be visible to the model
constrain what actions are permitted
execute tool-mediated work in a controlled sequence
handle recoverable failure without collapsing the session
stream results in a way humans can still follow

This is the first major lesson of the leak.

A mature agent is not just a prompt with tools attached. It is a managed action cycle.

7. The Strange Parts That Matter

Most leak coverage focuses on scale: file counts, line counts, directories, internal names. Those details are useful, but they do not all carry equal meaning.

Some parts matter more because they reveal design philosophy.

7.1 Permissions appear to sit inside the action loop

▫️Strong inference

One of the clearest signals from the analyzed material is that permissions do not appear to be a decorative security feature added around the outside. They appear to sit inside the execution cycle itself.

That distinction matters.

A superficial agent checks safety after something has already been decided. A more mature system checks the boundary at the moment action is about to happen.

The apparent design features include:

default-deny or fail-closed tendencies
tool-level and command-level distinction
escalation paths when certainty is insufficient
different persistence rules for different categories of approval

This should not be read as proof of a perfectly rigid system with no override logic. Real production systems often include bounded exceptions, onboarding shortcuts, or low-risk automation paths so that harmless actions do not create constant user friction.

The analyzed architecture also appears consistent with tiered approval logic, where lower-risk actions may pass through lighter-weight handling while higher-risk or more ambiguous actions trigger deeper review. The important question is not whether override paths exist. It is whether approval logic is tiered, scoped, and auditable.

That is what makes permissions here interesting. They look less like settings and more like runtime governance.

7.2 Persistent execution appears to be treated as infrastructure, not convenience

▫️Strong inference

Another revealing pattern is continuity. The analyzed material suggests the system is designed so that execution context does not reset after every action. That matters because a coding agent that forgets its shell state, working directory, and operational context after each step is not really working through a problem. It is repeatedly starting over.

This is where components like a persistent shell manager become meaningful. Without continuity, memory becomes mostly theatrical. With continuity, state begins to matter operationally.

That is an important architectural clue: session continuity appears to be a first-class design concern, not an afterthought.

7.3 Economic architecture appears to be part of the harness itself

▫️Strong inference

The analysis also suggests that Claude Code does not treat cost as a backend accounting detail. Stable prompt components and repeated context appear to be structured in a way that makes reuse more efficient across turns.

If that reading is correct, then the harness is not only managing intelligence and execution. It is also managing economics.

That matters more than it sounds.

Many teams still talk about cost optimization as something that happens after the agent works. A more mature design treats cost structure as part of the system architecture from the beginning.

A useful way to state the lesson is this:

A mature harness does not only manage intelligence. It also manages economics.

In real deployments, that can affect margins, latency, usage ceilings, and which products remain commercially viable at all.

7.4 KAIROS and similar names drew attention for a reason

▫️ Directional signal

Among the lower-confidence signals, one name attracted disproportionate attention: KAIROS.

It is important not to overstate what such names prove. A name alone is not a deployment guarantee. A suggestive code path is not the same thing as confirmed product behavior.

But the attention is still informative. What names like KAIROS seem to evoke is not merely curiosity about an internal feature. They point toward a broader architectural leap the market is actively trying to interpret: the move from a request-response assistant toward a more continuity-aware, background-capable, or always-on operating mode.

That distinction matters because it changes the engineering question. The issue is no longer only whether the model can answer well in one turn. It is whether the surrounding system can preserve state, revisit unfinished work, consolidate memory, and continue acting across time without losing control.

Even if public interpretations of KAIROS are uneven, the fascination around it still reveals something real. Attention is shifting from smarter outputs toward more persistent harness behavior.

That is a harness question.

8. Why This Changes the Conversation

The Claude Code leak mattered not only because of what it showed, but because of what it re-centered.

For the last phase of AI discourse, the center of gravity sat on model capability. Bigger context windows. Better benchmarks. More persuasive demos.

The leak pushes attention somewhere else.

▫️Strong inference

The deeper lesson is that production systems succeed or fail on questions the model alone cannot answer:

What information is injected, and when?
What tools are available, and under what conditions?
What happens when a step fails halfway through?
What persists across turns or sessions?
How are costs controlled as work continues?
How much of the system remains legible to a human operator?

These are not model questions. They are harness questions.

That is why this incident may matter beyond Anthropic.

It did not merely expose one company's code. It helped clarify where the next engineering competition is likely to sit.

9. From Agent Talk to Harness Thinking

This is where the report moves from analysis of one event to a broader field-level thesis.

The word agent has done useful work. It gave the industry a way to talk about systems that do more than answer prompts.

But the word also hides too much.

It can refer to a simple loop with tools, a multi-step code assistant, a background worker, a policy-bounded coordinator, or a semi-persistent execution environment. The label is broad enough that it often obscures the actual engineering surface.

▫️This report's argument

The more useful design question is no longer simply:

How autonomous is the agent?

It is increasingly:

What kind of harness turns that autonomy into bounded, observable, durable action?

This is why the leak matters as an architectural event. It gives substance to a transition that was already beginning: from talking about agents as if they were personalities, to talking about harnesses as if they were systems.

That change in vocabulary is not cosmetic. It reflects a change in engineering focus.

10. The Next Problem: Drift

Once the harness becomes responsible for context, permissions, continuity, memory, and cost structure, the next unresolved question is not simply whether the system works today.

It is whether the system remains safe and meaningful over time.

That is the drift problem.

▫️Plain-language definition

Drift is what happens when a system that was once safe, correct, or well-bounded becomes less so over time — not because one visible catastrophic change happened, but because meanings, dependencies, approvals, and contexts slowly moved.

Examples are easy to imagine:

a previously harmless command now touches a more sensitive environment than before
compressed memory silently drops context that later becomes important
a model update changes how a familiar boundary is interpreted
an old approval is reused in a new situation where its meaning has shifted

A concrete software example makes the problem clearer. Imagine a tool call that once wrote to a benign local artifact, and was therefore approved as low-risk. Months later, a dependency update changes the output schema, or the same path is now wired into a deployment-sensitive pipeline. The approval may still exist, but the meaning of the action has changed. That is drift.

▫️Why drift belongs in this report

This is not a separate manifesto topic added from outside the leak.

It follows naturally from the kind of architecture the leak appears to reveal. Once a harness governs repeated action across time, the problem is no longer just execution quality. It is temporal reliability.

▫️Strong inference

The analyzed Claude Code material appears comparatively mature in orchestration, permissions, continuity, and cost awareness. It does not appear to offer an equally mature public answer to long-horizon drift-aware revalidation.

That may be one of the most important unresolved engineering problems in this entire area.

The field is getting better at making agents act.

It is less clearly getting better at making yesterday's safe behavior remain safe tomorrow.

11. What This Leak May Have Changed for the Field

A single leaked codebase does not settle the future of AI architecture. But some events clarify what many people were already beginning to suspect.

This may be one of those events.

▫️This report's synthesis

The Claude Code leak suggests five field-level conclusions:

The model alone does not explain product performance.
The surrounding harness appears to carry more of the product than many people assumed.
Governance is becoming executable.
Permissions, approval boundaries, traceability, and action control are increasingly engineering concerns, not only policy language.
Economics is now architectural.
Cost control is not just a billing issue. It is part of how serious systems must be designed.
Continuity matters as much as intelligence.
An agent that cannot preserve state, context, and working continuity is limited even if the underlying model is strong.
Drift may be the next meaningful frontier.
The industry has become stronger at orchestration than at long-horizon revalidation.

Those conclusions should not be treated as universal law. But they are, at minimum, a defensible reading of why this leak produced so much architectural interest.

12. Conclusion

The Claude Code leak was embarrassing for Anthropic. That much is obvious.

What is less obvious, and more important, is what the event revealed about the real engineering surface of modern AI systems.

The strongest lesson is not that the model does not matter. It does.

The stronger lesson is that model capability alone does not explain production performance. The harness — the software layer that manages context, tools, permissions, recovery, continuity, and economics — appears to carry far more of the product than many people had previously acknowledged.

That is why this incident matters beyond one company and beyond one week of coverage.

It helps clarify a broader shift:

from prompt tricks to managed execution
from raw autonomy to governable action
from “agent” as a label to “harness” as the real engineering surface

The next durable moat in AI may not be the model alone.

It may be the harness that turns model capability into bounded, observable, durable work.

Coder Legion - Recent posts tagged harness

The human defines, delegates, reviews, and is responsible. The AI is not.

Key points

The gate that wasn't

"Keep a human in the loop" is more rhetoric than practice

The wrong frame

Author, judgment, conscience

The harness

The point is not to remove the human

Further reading

The Agent Moved In. Code Is Now the Address.

Anatomy of a Coding Agent: What I Learned Building My Own

The Big Picture

The Agent Loop

The System Prompt

Tools: The Model’s Hands

Memory & Context Management

Policy & Approvals

Local-Model Survival

Provider Abstraction

The Extension Surface

What I Left Out

What This Means

The Harness Is the Product: What the Claude Code Leak Actually Revealed About AI Agent Architecture

Executive Summary

1. Why This Leak Mattered More Than Most

2. Reading Guide: What This Report Claims, and What It Does Not

▫️Confirmed

▫️Strong inference

▫️Directional signal

▫️What this report does not claim

3. Five Terms in Plain English

4. What Happened — and What Did Not

▫️Confirmed

▫️Why this mattered immediately

▫️Strong inference

5. The First Real Surprise: The Model Was Not the Whole Product

▫️Strong inference

6. The Core Pattern: A Managed ReAct Loop

▫️Strong inference

7. The Strange Parts That Matter

7.1 Permissions appear to sit inside the action loop

▫️Strong inference

7.2 Persistent execution appears to be treated as infrastructure, not convenience

▫️Strong inference

7.3 Economic architecture appears to be part of the harness itself

▫️Strong inference

7.4 KAIROS and similar names drew attention for a reason

▫️ Directional signal

8. Why This Changes the Conversation

▫️Strong inference

9. From Agent Talk to Harness Thinking

▫️This report's argument

10. The Next Problem: Drift

▫️Plain-language definition

▫️Why drift belongs in this report

▫️Strong inference

11. What This Leak May Have Changed for the Field

▫️This report's synthesis

12. Conclusion