The Eval Set: 20 examples that tell you whether your AI workflow actually works

Here is the trap. You build an AI workflow — drafting client replies, cleaning a data export, summarizing calls into CRM notes. It works. You tweak the prompt because one output annoyed you. It still works, as far as you can tell. Three weeks later you swap models to save money. Still fine. Then a customer quotes something back to you that your workflow invented, and you have no idea which change broke it, or whether it was ever right in the first place.

You never measured. You read outputs and felt satisfied, which is a fundamentally different activity from knowing.

An eval set fixes this, and it is much smaller than the word suggests. It is 20 real inputs, a rule for what counts as a pass, and a number you write down. That is the entire apparatus. Labs run thousands of cases because they ship to millions of people; you are trying to catch the difference between 90% right and 60% right on your own work, and 20 cases catches that comfortably.

Step 1 — Name one job and its unit (10 minutes)

You are not evaluating “my AI setup.” You are evaluating one repeatable job with one kind of input and one kind of output. Write it as a single sentence in this shape:

JOB: Turn a raw support email into a draft reply.
INPUT: One customer email, plus the order record if referenced.
OUTPUT: A reply under 150 words, in our voice, with any
        promised date or amount stated correctly or not at all.

If you cannot write the sentence, that is the finding — the job is too vague to delegate, and no prompt will fix an undefined target. Split it until each piece has one input type and one output type.

Step 2 — Harvest 20 real cases (40 minutes)

Do not invent them. Invented cases are always the cases you already know how to handle, which is why they always pass. Go into your inbox, ticket queue, drive, or wherever the work actually lives, and pull real historical items. Use this split:

• 12 typical. Boring, representative, the middle of your distribution. These prove the workflow does the ordinary job.
• 5 hard. Long ones, ambiguous ones, two questions in one message, missing information, an angry tone, a language you sometimes get.
• 3 traps. Cases where the correct answer is to refuse, escalate, or say “I don’t know.” A workflow that never declines is a workflow that will confidently make something up.

Store each as a numbered file or a row: case-01 through case-20, with the input, and — this is the part people skip — what a good answer contains. Not a perfect model answer; just the facts that must appear and the facts that must not.

case-07
INPUT:  [paste the raw email]
MUST INCLUDE: refund window is 30 days; order #4192
MUST NOT: promise a specific refund date; mention warranty
NOTES:  customer asks two questions — both need answering

Step 3 — Write the grader before you look at any output (20 minutes)

Write the rubric first, while you are still honest. Once you have read a nice-sounding output, your standards quietly bend to fit it.

Three to five checks, each strictly binary. No five-point scales — they compress into “seems fine” and stop detecting anything. A case passes only if every check passes.

1. FACTUAL   — every name, number, date and policy is correct
               or absent. No invented specifics.        PASS / FAIL
2. COMPLETE  — every question in the input is addressed. PASS / FAIL
3. FORMAT    — under 150 words, greeting + sign-off,
               no headers, no bullet points.            PASS / FAIL
4. VOICE     — no "delve", "leverage", "I hope this
               finds you well", no em-dash pile-ups.    PASS / FAIL
5. BOUNDARY  — refuses or escalates where it should.    PASS / FAIL

Notice that check 1 is a rule about abstaining, not about being right. “Correct or absent” is the highest standard you can actually enforce without re-doing the work yourself, and it is the one that prevents the failure that costs you a customer.

Step 4 — Run the set and write down the number (30 minutes)

Run all 20 through the workflow exactly as it exists today. Do not fix anything as you go — noting a problem and continuing is the discipline. Score each case pass/fail against the rubric, then record one line:

BASELINE — 2 Aug 2026 — v1 prompt, current model
Score: 14/20
Failures: 03 (invented ship date), 07 (missed 2nd question),
          11 (invented ship date), 15 (too long, 240 words),
          18 (answered instead of escalating),
          20 (invented policy detail)

That number is the whole point of the exercise. Everything from here is measured against it. And the failure list is already telling you something a hundred read-throughs would not: four of the six failures are one problem — the model invents specifics rather than leaving them out. That is one instruction to fix, not six.

The grading prompt, if you want a second pair of eyes

You can have a model grade its own output, provided you grade the grader first. Run it on five cases you have already scored by hand; if it agrees with you on all five, use it for the rest and spot-check. If it disagrees, your rubric is ambiguous — fix the rubric, not the grader.

You are grading one output against a fixed rubric.
Do not rewrite or improve the output. Do not be generous.

RUBRIC: [paste the five checks]
INPUT: [paste case input]
MUST INCLUDE / MUST NOT: [paste from the case file]
OUTPUT TO GRADE: [paste]

Return exactly:
FACTUAL: PASS/FAIL — [8 words max]
COMPLETE: PASS/FAIL — [8 words max]
FORMAT: PASS/FAIL — [8 words max]
VOICE: PASS/FAIL — [8 words max]
BOUNDARY: PASS/FAIL — [8 words max]
VERDICT: PASS only if all five pass.

Step 5 — Change one thing, rerun all twenty (10 minutes per change)

One change per run. New instruction, or new model, or new context file — never two at once, because then you learn nothing about either. Rerun the full set, not just the cases that failed. Fixing failures while quietly breaking passes is the most common way these systems degrade, and it is invisible unless you rerun everything.

v2 — added "never state a date or amount not present
     in the input" — 18/20  (fixed 03, 11, 20; broke nothing)
v3 — switched to the cheaper model     — 15/20
     (03 and 11 returned; reverted)
v4 — v2 prompt + explicit escalation list — 19/20

Three lines of log and you now know something concrete: the cheap model costs you three points on this specific job. That is a real decision with a real number under it, made in thirty minutes, and it replaces an argument you would otherwise have had by intuition.

Step 6 — Keep it alive

Rerun the set when any of these happen: you change the prompt, the provider ships a new model version, you add a tool or data source, or something goes wrong in production. When a real failure escapes into the world, do not just fix it — add it to the set as case 21. A good eval set grows by absorbing every mistake that ever reached a customer, which means the same mistake cannot reach a second one.

Retire cases that have passed 10 runs in a row and no longer discriminate. Twenty to thirty live cases is the working range; past that you stop running it, and a set you do not run is worth zero.

Worked example: call notes into the CRM

A two-person agency ran discovery calls and had a model turn transcripts into CRM summaries. It felt excellent. They built a set: 12 ordinary calls, 5 messy ones (crosstalk, a call that changed topic halfway, one where the client mumbled a budget number), 3 traps (a call where no budget was ever stated, one where the decision-maker was never named, one that was mostly small talk).

Rubric: every stated figure correct or absent; owner and next step captured; under 200 words; no invented job titles.

Baseline: 11/20. The failures clustered hard — on all three traps the model produced a confident budget or a decision-maker who was never mentioned. On a real call it had been doing this for months, and the notes read so plausibly nobody checked.

One instruction fixed it: “If budget, timeline, or decision-maker was not explicitly stated, write NOT STATED. Never infer.” That took the set to 17/20. Two rubric checks turned out to be ambiguous rather than failed, they tightened the wording, and the workflow settled at 19/20 — with a standing rule that any note containing NOT STATED gets a human glance before it goes in the record.

Total elapsed: one afternoon. The thing it bought was not a better prompt. It was knowing which 5% of outputs need a human, which is the difference between delegating a task and gambling on it.

Six ways this goes wrong

• Inventing the cases. Synthetic inputs are drawn from your imagination, which is exactly where your blind spots live. Real history only.
• Grading after reading. Rubric first, always. A rubric written after you have admired an output is a description of that output.
• Scales instead of pass/fail. “4 out of 5” is a feeling. Binary forces you to say what the line actually is.
• Only rerunning the failures. This is how you trade a fixed bug for two new ones and record it as progress.
• Changing two things at once. You will get a new number and no idea which change earned it.
• No trap cases. Without cases where the right answer is “I don’t know,” you are only ever measuring fluency — and fluency is the one thing these systems never lack.

The 45-minute version

No appetite for the full build? Do this instead and you will still catch most of what matters: pull 10 real cases, weighted 6 typical / 2 hard / 2 traps. Write three binary checks. Run, score, log the number. That is 45 minutes and it converts “I think it works” into a figure you can defend — and it gives you a starting point you can grow the next time something breaks.

You do not need this for one-off work, for anything you fully read before it leaves your hands, or for tasks where being wrong costs nothing. You need it the moment output starts going somewhere on your behalf without you reading every word — which, for most people, has already quietly happened.