Advanced testing
Prove the change beat the noise.
A rubric score that moves two points has told you nothing until you know how much it moves on its own. Rubrkit audits both versions repeatedly and reports whether the difference cleared the evaluator's own spread.
verdict better
overall +12.4 CI [8.1, 16.7] p < 0.0001
clarity +0.9 CI [0.61, 1.27] p 0.0004
output format +1.3 CI [0.98, 1.64] p < 0.0001
self-contained -0.1 CI [-0.44, 0.28] not significantThe same text, graded twice.
Nothing changed between these two runs — same artifact, same rubric, same model. The gap between them is the bar every real improvement has to clear, and it is the number most tools never show you.
run 1
84
run 2
86
no significant change — a 2 point move, correctly called nothing
drift check · pinned v1 · weekly · 3 audits
baseline 83.3 sd 3.06 (calibrated, n=3)
normal range 74.2 — 92.5
this check 85.3 flat · within bandA number with its error bars attached.
A single score hides its own variance, which is exactly what makes it feel authoritative. An advanced comparison audits both versions several times and reports the difference against the spread of the evaluator itself.
Repeated audits, both sides
Each version is graded several times, so the comparison has a distribution to work with instead of two lucky samples.
A confidence interval per dimension
Every rubric dimension gets its own interval drawn against zero. A bar that crosses the centre line is not a small win — it is not a win.
Corrected for ten dimensions
Testing ten things at once manufactures false positives. A Benjamini-Hochberg correction is applied across dimensions before anything is called significant.
A golden set that grows
Pin the real inputs this artifact must keep handling. Every failure you find becomes a case, so the set sharpens instead of staying theoretical.
Drift on a pinned version
Re-audit one frozen version on a schedule. The artifact cannot have changed, so movement outside its calibrated band means the model or the rubric did.
Claims it can actually support
This measures rubric-judged quality across repeated audits, and says so wherever a verdict appears. Real-user significance needs a labeled set or live outcomes.
Bring Rubrkit into your toolchain.
Sync graded artifacts into the repos where your agents run, or connect any MCP client to the same audits, bundles, and rubr_flow tools the web app uses.
CLI
Pull approved artifact bundles into local projects and place them where Codex, Claude, or generic agents expect their instructions.
# Pull approved artifacts into your project npx rubrkit pull # Place them where your agent expects them npx rubrkit pull all --agent claude --yes
MCP server
Point any MCP client at Rubrkit and call the same artifact bundle, audit, and rubr_flow tools, authenticated with your Rubrkit API key.
{
"mcpServers": {
"rubrkit": {
"url": "https://www.rubrkit.com/api/v1/mcp",
"headers": {
"Authorization": "Bearer <your-rubrkit-api-key>"
}
}
}
}Answers before you start.
Know which instructions are ready to run.
Compare two versionsFollow the review loop as it ships.
Notes on AI artifact testing, rubr_flow conversion, evals, and proof reports.