Check active goals against rubric
Verify each goal in active.md has evidence attached, stopping after 25 turns or completion.
/goal all rubric items in .ultragoal/goals/active.md are checked with evidence, or stop after 25 turns
claude-code
More evaluation loops
Attack a design until it holds
A critic hammers the design and a builder answers — every objection tracked, and none closed without evidence.
Before committing to an architecture, interface, or rollout plan, have a critic argue that it is wrong. Record each objection, impact, and status in a repository-local log at .agent-reviews/redteam.md. The builder must fix and verify each high-impact weakness or document why it is accepted; the critic may reopen unsupported answers. Stop when no high-impact objection remains or the same issues repeat for two rounds without new evidence. Finish with the decision, resolved and accepted objections, evidence, and any stalemate.
evaluationmedium risk
Two models must agree
Alternate two models from different providers to review a plan, doc, or diff until both approve the exact same version.
Review [plan, specification, document, or code change] against [quality bar] for at most [pass limit] rounds. Have one of two genuinely different model families—AI systems from separate providers—review it. Verify each finding and apply only necessary fixes, then give the revised version to the other reviewer. Succeed only when both approve the same unchanged version. Stop at the limit, repeating disagreement (oscillation), unavailable review, or required approval. Return the final work, round log, verdict, and disagreements.
evaluationmedium risk
Promote prompts only on holdout wins
Test challenger prompts on a working set, promote only on fresh holdout wins, and keep the champion when results are uncertain.
Improve a prompt, policy, or configuration. A support assistant's system prompt is one example. Save the champion, its score, a working set, untouched holdout cases, must-pass checks, and [budget]. Each round, change one thing based on a recorded failure. Promote the challenger only if it beats the champion on holdouts by [margin] without weakening a must-pass check; otherwise keep the champion. Stop at the target, budget limit, or no progress. Return the winner, scores, experiment log, and remaining failures.
evaluationmedium risk