Notes from the review loop.
How we think about grading AI instructions against a rubric, writing rewrites you can test, and versioning prompts, agents, and skills like the production code they are.
How to test a prompt: a practical guide to prompt regression testing
A working procedure for testing prompts the way you test code — a golden set, pass criteria you can defend, and a CI gate that catches the regression before your users do.
LLM evaluation metrics: what to measure and when each one lies to you
Exact match, BLEU, embedding similarity, LLM-as-a-judge, rubric scores. Each measures something real and each fails in a specific, predictable way. Here is which to reach for and what it hides.
How to get statistical significance for your prompts
The honest answer has two halves: what real significance costs, and what you can get for far less. Both start by measuring how much your evaluator moves when nothing changes.
The ten dimensions we grade every instruction on
One number is easy to argue with. Ten named dimensions are not. Here is the rubric behind every Rubrkit score, and what each one is actually checking for.
Ship the proof, not the promise
"I improved the prompt" is a claim. A proof report is evidence — the same instruction, before and after, with the eval it now passes and the version hashes to prove which is which.
When your prompt is really a workflow
Some prompts are workflows wearing prose. "Do this, then that, branch if X" doesn't belong in a paragraph. Here is how to convert one into a bounded flow you can grade as steps.
How we grade our skills with CI
We open-sourced a library of skills — then wired our own quality gate to grade them on every push. A skill that drops below the bar turns the build red.
Audit the prompt before you run the agent
A long prompt is cheapest to fix before it runs. Here is how to grade and harden an instruction over MCP, in the same client your agent lives in, before you hit go.
Stop grading prompts on vibes
A prompt that "feels good" is not a prompt you can trust. Here is why we grade instructions against a rubric instead of a gut feeling.
Anatomy of a testable rewrite
A weak instruction, the rubric dimensions it fails, the rewrite that fixes them, and the eval that proves the rewrite holds.
Version your prompts like code
Prompts, agents, and skills are production instructions. They deserve history, provenance, and a single source of truth — not a paste buried in a chat log.