How we grade

Safety Score v1.0 · documented from the shipped grader (score.mjs) · weights verified against the live system with 16/16 probes, 27 July 2026

Every A–F on this site comes from one deterministic function. The same code grades the directory, the /grade page, the MCP loops_grade tool, and the ingest pipeline. No model judgment is part of the grade — the AI take on /grade is advisory and clearly labeled. This page is the whole rubric, including the parts that are weak. If a grade surprises you, the explanation is below, not hidden.

Two verdicts, two authorities

A graded loop gets two independent verdicts. The gate (pass/reject) comes from the default-fail evaluator that guards the directory: dangerous commands, spam, and structurally unbounded loops are rejected, and unclear cases reject by default. The score (A–F) is a separate weighted checklist of harness signals. The gate is the authority on publication; the score is the authority on nothing except itself — it is a harness-quality summary, not a permission slip.

Weights

Signal	Points	What fires it
Passes the danger + spam screen	+40	No destructive command (rm -rf, force-push to main, pipe-to-shell, permanent mail deletion) and no spam markers. This is the largest single award in v1 — see limits below.
Exit condition	+20	A framing word (until / stop when / exit when / stop after N) plus a target. Soft targets count in v1: "until done" and --completion-promise both fire this signal.
Iteration cap	+15	A numeric budget: "stop after 10 turns", "max 15 iterations", --max-iterations 50, MAX_ITER=20.
Verification	+10	A named check: tests, lint, build, typecheck, CI, "exits 0". The bare noun "tests" is enough in v1 — it does not require the check to be executed.
Scoped	+10	Anti-reward-hacking language: "only", "never touch/modify/delete", read-only, report-only, minimal diffs.
Human gate	+5	Ships through review: PR, propose, draft, approval.

A danger-pattern hit caps the total at 10 regardless of other signals. A spam hit removes the +40. Maximum 100.

Grade bands

Grade	Score	What it takes in practice
A	90–100	Clean screen, an exit condition, and a cap are mandatory — missing any of the three caps you at 85. Those reach 75; an A needs 15 of the remaining 25 points (verification, scope, human gate).
B	75–89	Clean screen plus most of the harness signals.
C	55–74	Clean screen plus roughly two signals.
D	35–54	The floor for any prompt that passes the danger/spam screen. A bare unbounded loop lands here, not lower.
F	0–34	In practice: a danger-pattern hit (score capped at 10) or spam. Weakness alone cannot produce an F in v1 — see limits.

Known limits of v1.0

These are not hypotheticals — each was verified against the live grader during the 27 July 2026 calibration. We publish them because a rubric you can't audit is just a vibe with a letter attached.

F is unreachable through weakness alone. A prompt that passes the danger/spam screen starts at 40 points — already a D. A literal unbounded "while true" loop with no exit, no cap, and no verification grades D·40, not F. In v1, F means "matched a danger pattern or spam", never "structurally unsafe".

Most of the passing score is for not matching a regex. 40 of the points are awarded for the absence of a destructive string. Absence of evidence is weak evidence: a prompt can avoid every pattern we check and still be a bad idea.

The biggest positive signal accepts soft promises. The +20 exit-condition signal fires on "until done" and --completion-promise — self-attested completion, the exact thing our own listings tell you not to trust. A machine-checkable exit ("until npm test exits 0") and a model promise are worth the same in v1.

The danger gate is a narrow regex. It catches rm -rf, force-push with a literal branch name, curl|sh, and permanent mail deletion. It does not catch --force-with-lease, prose phrasings ("force-push the result"), or anything we have not enumerated. Passing the gate is not a safety certificate.

Two doors, two answers. Directory grades are computed from the prompt field alone; /grade scores whatever text you paste. Pasting description + prompt moved 7 of 52 ralph-type rows exactly one grade letter in the 27 July regrade. v1 has no canonical input.

No recovery signal. A loop that classifies failures and replans scores identically to one that blindly retries. v1 cannot see the difference.

What's next — v2 (spec, not live)

A v2 spec exists (27 July 2026) and is calibrated against every published row. Nothing in it is live; v1.0 still grades this directory. The design directly targets the limits above:

One canonical input everywhere — description + prompt — closing the two-doors gap.
Passing the danger screen drops from +40 to +15; machine-checkable exits (+25) split from soft promises (+5).
New signals: recurrence/wall-clock bounds (+15), executed verification (+15) vs merely named (+8), recovery behavior (+10).
A structural F: unbounded + self-attested completion + no executed verification fails without needing a danger string.
Projected effect on the corpus at spec time: A count drops 40 → 16, F rises 0 → 31. Grades will get harder, and that is the point.

When v2 ships, grades will carry a rubric version stamp so a v1 A is never silently compared to a v2 A.

Grade your own loop →