Evals Are the Only Honest Signal You Have

Most teams evaluate LLM changes by trying a few prompts by hand and forming an impression. It feels efficient and it works early, when the system is small and you remember every case. It fails in a specific and expensive way: you fix a bug, something unrelated regresses, and you find out weeks later from a user.

The reason manual checking breaks down is that the output space is enormous and your sample is tiny and biased. You test what you just changed. The regression is somewhere you were not looking.

What an eval actually is

An eval is a case plus a programmatic verdict. The case is an input and whatever context it needs. The verdict is code that decides pass or fail without a human reading the output. If a person has to read it, it is a review, not an eval, and it will not run on every commit.

The verdict does not need to be clever. Most useful evals are boring assertions.

// Schema validity: cheap, catches a real class of breakage.
expect(() => InvoiceSchema.parse(output)).not.toThrow();

// Behavioural: the right tool, and none of the dangerous ones.
expect(toolsUsed).toContain("searchInvoices");
expect(toolsUsed).not.toContain("sendEmail");

// Grounding: every cited id exists in the supplied context.
for (const id of citedIds) expect(contextIds).toContain(id);

// Budget: no silent 10x cost regression.
expect(totalTokens).toBeLessThan(20_000);

None of that judges writing quality, and that is fine. These four categories catch the majority of real regressions, they never flake, and they cost nothing to run.

Use a model as judge carefully

For subjective dimensions like tone, helpfulness or whether an explanation is clear, you can have a model grade the output. It is useful and it is also the easiest part of an eval suite to get wrong.

• Grade one narrow dimension per judge call; a single 'is this good' score is noise.
• Give the judge an explicit rubric with concrete examples of each verdict, not an adjective.
• Ask for a short justification before the verdict, which measurably improves consistency and it makes disagreements debuggable.
• Validate the judge against human labels on a sample. An unvalidated judge is a number that feels like evidence.

Keep deterministic assertions as your gate and model judgement as a trend line. Blocking a deploy on a fuzzy score you have not calibrated trades a real problem for a flaky one.

Start with ten cases today

The reason most teams have no eval suite is that a proper one sounds like a quarter of work, so it never starts. It is not. Take the last ten things that went wrong, write them down as inputs with an assertion each, and run them in CI. That suite is already more signal than the impression you are currently working from.

Every production bug should end as a permanent case. That is the whole discipline, and it compounds.

The payoff is not that quality goes up immediately. It is that quality stops silently going down, and you regain the ability to change the system confidently. Without evals, every prompt edit is a gamble you cannot price. With them, shipping an AI feature starts to feel like ordinary engineering again.