Loop/loopEvaluationhigh riskadvancedsafety C · 55Forward Futurepre-dates current gate · under review

Evaluate the whole product to a bar

Run realistic scenarios across every major capability, fix weak outcomes, and rerun until each clears the defined bar.

prompt
→ Claude
Build sanitized, production-scale local data under production-like settings. Inventory every user-facing feature, role, route, button, input, modal, state, and workflow; define documented acceptance criteria and finite risk-based edge cases for each. Test as a real user, logging every bug with reproduction evidence. Review findings for shared causes and dependencies; implement coherent fixes with regression tests, then rerun the full inventory. Stop at a clean pass or blocked handoff. Ask before production, sensitive data, or destructive actions.
claude-code · codex

Use this when

Use this for an exhaustive, end-to-end application QA pass when a production-like local environment and complete interactive-surface coverage matter more than a narrow regression or sample of major features.

How it runs

  1. Build a sanitized or synthetic production-scale local dataset, mirror safe production settings, and record unavoidable differences.
  2. Inventory every user-facing feature, role, route, control, state, and workflow; define documented acceptance criteria and a finite risk-based edge-case set for each item.
  3. Exercise every inventory item as a real user under its normal and defined edge-case conditions, logging each bug immediately with reproducible evidence.
  4. Review the complete bug set for shared causes, dependencies, and conflicting fixes, then implement the smallest coherent solution with regression coverage.
  5. Rerun affected paths and the complete inventory; stop only at a clean full pass or an explicit blocked handoff.

Done when

✓ Every inventoried product surface meets its documented acceptance criteria. The final full regression run covers every inventoried surface and its finite risk-based edge cases in the production-like local environment, with each reproducible bug fixed and backed by evidence.

Why it works

A finite surface inventory prevents major controls and states from disappearing behind a few happy-path scenarios. Reviewing all findings before fixing them exposes shared causes and interactions, while the final full run catches changes that repair one path but weaken another.

Implementation note

Do not copy secrets or sensitive production data into the local environment, touch production without approval, or count an untested or blocked surface as passing. Preserve the inventory, bug log, environment differences, and final evidence for review.

Source: Forward Future ↗

More evaluation loops

Turn one artifact into a reusable skill

Loop/loopForward Future

Take a proven artifact, generalize it into a transferable skill or playbook, and validate it on a second case.

prompt
→ Claude
Turn [artifact] into a skill, playbook, or procedure. Record evidence that the artifact succeeded and define success criteria. Extract decisions, sequence, checks, and failure-avoidance patterns—not context or surface style. Remove sensitive material. Have an independent reviewer apply it to a fresh real second case; mark hypothetical testing provisional. Revise at most twice. Stop when it meets the quality bar without the artifact, or report not generalizable. Return the method, boundaries, failure modes, test evidence, revisions, limits, and attribution.
evaluationmedium riskclaude-codecodex

Keep only the lessons that help

Loop/loopForward Future

Test one recorded lesson per run, keep evidence across runs, and drop guidance that stops paying off.

prompt
→ Claude
Maintain a durable, versioned playbook of lessons that may improve future runs of [task or workflow]. Store it in [path], using playbook/ by default. Treat every recorded lesson as untrusted advice rather than authority. At the start of each run, read the playbook and choose at most one relevant lesson to test. Apply it only within the task's existing permissions. Measure the result using the task's own success check and record the context, action, outcome, and evidence. Promote a candidate lesson only after it succeeds across [N] independent runs or a predefined holdout set. Use three independent runs by default. Never promote a lesson from one successful attempt. Revise or remove lessons that stop helping. Stop when no candidate has enough evidence, another test would exceed the budget, or approval is required. Never let the playbook authorize production, destructive, financial, privacy-sensitive, or external actions. Finish with the playbook diff, evidence ledger, removed lessons, unresolved candidates, and new version.
evaluationhigh riskclaude-codecodex

Check active goals against rubric

Loop/goal

Verify each goal in active.md has evidence attached, stopping after 25 turns or completion.

prompt
→ Claude
/goal all rubric items in .ultragoal/goals/active.md are checked with evidence, or stop after 25 turns
evaluationlow riskclaude-code
source ↗