Loop Engineering: Designing Agent Control Flow

Strip the vocabulary away and an agent is a loop: call the model, run whatever tool it asked for, feed the result back, repeat until something says stop. The interesting engineering is entirely in that last clause. Most agent problems I have debugged were not reasoning failures. They were control-flow failures in a loop nobody designed on purpose.

The shape of the loop

let messages = [systemPrompt, userGoal];

for (let step = 0; step < MAX_STEPS; step++) {
  const res = await model.generate({ messages, tools });

  if (res.stopReason !== "tool_use") break;   // model is done

  const results = await runTools(res.toolCalls);  // may fail
  messages = compact([...messages, res, ...results]);

  if (budgetExceeded() || noProgress(step)) break;  // we are done
}

Fifteen lines, and four of them are termination conditions. That ratio is not accidental. Deciding when to stop is most of the work.

Both failure modes are expensive

Loops fail in two opposite directions and the fixes pull against each other, which is why naive tuning oscillates.

• Stopping early: the agent reports success having done part of the job. Costs you trust, and it is worse than an outright error because it looks like completion.
• Never stopping: the agent relitigates the same step, re-reads the same file, tries a variant of a call that already failed. Costs you money linearly and reveals itself only on the bill.

A hard step cap bounds the damage from the second but does nothing for the first, and if you set it too low you manufacture the first. The cap is a safety net, not a design.

Make progress explicit

Spinning happens because nothing in the loop distinguishes a step that advanced the task from a step that merely consumed tokens. So measure it. Before each iteration, record a cheap fingerprint of what the agent is attempting, the tool plus its salient arguments, then compare that against recent history.

const fp = `${call.name}:${stableHash(call.args)}`;

if (recent.filter((h) => h === fp).length >= 2) {
  // Third identical attempt: the loop is not converging.
  // Inject the observation rather than silently looping again.
  messages.push({
    role: "user",
    content:
      `You have already attempted ${call.name} with these arguments twice ` +
      `and it did not succeed. Do not repeat it. Either try a materially ` +
      `different approach or explain what is blocking you.`,
  });
}

recent.push(fp);

Naming the repetition back to the model works remarkably well, and it is strictly better than killing the run. A hard stop gives the user nothing; this usually gets either a genuine alternative approach or a clear statement of the blocker, which is itself actionable.

Tool failures are information, not exceptions

The instinct from ordinary backend work is to catch an error and abort. In a loop that is usually wrong. A failed tool call is a legitimate observation the agent can act on. A missing file means look elsewhere; a validation error means fix the arguments. Feed the error back in full and let the next iteration use it.

What you must not do is swallow it. An empty tool result with no explanation is the single most confusing thing you can hand a model: it cannot distinguish 'no matches exist' from 'your query was malformed' from 'the service is down', so it guesses, and the guess drives the next ten steps.

Budget in money, not iterations

Step count is a poor proxy for cost, because a step can be a two-hundred-token confirmation or a hundred-thousand-token file read. Track cumulative tokens and wall-clock time, enforce a ceiling per run, and surface both in your traces. Then set the ceiling from what the task is worth, not from a number that felt safe.

The loop is the product. The model is a very good function you call inside it.

None of this is exotic. It is idempotency, backoff, budgets and observability, the same concerns as any distributed system. The discomfort is only that one node in the graph is probabilistic, which means you design for it being wrong sometimes rather than hoping it will not be.