Harness Engineering: The Real Work Is Around the Model

Swap the model behind a working agent and it usually still works. Swap the harness and nothing works. That asymmetry tells you where the engineering actually lives, and it is not in the part everyone talks about.

By harness I mean everything wrapped around the inference call: the tools you expose and how you describe them, what the agent is allowed to do without asking, how state survives between steps, what you log, and how you stop a run that has gone wrong. This is ordinary systems engineering. It is also where essentially every production failure I have debugged originated.

Tool design is API design for a fallible caller

A tool definition is a public API whose only consumer is a probabilistic client that cannot read your source, will not remember your conventions, and will confidently guess when your description is vague. Every ambiguity you leave becomes a malformed call.

The failure mode I see most often is tools that mirror internal database shape rather than the task. A tool taking fourteen optional parameters, eleven of which are only valid in combination, will be called wrong. Not because the model is weak, but because the contract is genuinely unclear.

// Weak: leaks internal shape, ambiguous units, silent failure surface.
search({ q, filters, opts, page, limit, sort, dir, mode })

// Stronger: task-shaped, closed vocabularies, explicit units.
searchInvoices({
  customerId: string,
  status: "paid" | "overdue" | "draft",
  issuedWithinDays: number,   // NOT "range"
  limit: number,              // max 50, documented
})

Two rules pay for themselves immediately. Prefer closed enums over free-form strings, because an enum is a constraint the model cannot violate. And make error returns instructive rather than terse: 'limit must be between 1 and 50, received 500' produces a correct retry, while 'invalid request' produces a random one.

Read freely, write under permission

The single most useful structural decision is splitting tools by reversibility rather than by domain. Reads are cheap and recoverable, so let the agent do as many as it wants. Writes that touch the outside world, like sending, publishing, deleting or paying, are a different category and belong behind an explicit gate.

• Unrestricted: queries, reads, computation, anything you can run a thousand times with no consequence.
• Gated: mutations with blast radius, such as outbound messages, destructive deletes, financial movement and public writes.
• Never automated: credential handling and irreversible bulk operations, regardless of how confident the agent sounds.

This boundary is also what makes autonomy safe to increase. You can let a well-instrumented agent run for hundreds of read steps without supervision precisely because the destructive surface is small, enumerated, and gated.

You cannot debug what you did not record

Agent bugs are not reproducible from a stack trace. By the time you see a bad outcome the decision that caused it happened fifteen steps earlier, and the only artifact that explains it is the exact context and tool I/O at that step. If you did not persist it, the bug is effectively unfixable, and you are guessing.

• The fully-assembled prompt per step, not a template plus variables you hope you can re-expand.
• Every tool call with its literal arguments and its literal response, including failures.
• Token counts and latency per step, so runaway cost shows up as a graph instead of an invoice.
• A stable run id threaded through everything, so one user complaint maps to one exact trace.

Treat an agent run like a distributed transaction: if it is not traced, it did not happen in a way you can reason about.

The harness is your moat

Model capability is converging and getting cheaper, and any advantage you get from picking this quarter's best model is rented. The tool surface that matches your domain, the permission model your compliance team accepts, the traces that let you fix a regression in an afternoon. Nobody can copy those by swapping an API key. That is the durable part, and it is worth building deliberately.