API contract test backfill
Generate a contract test for every documented endpoint in the OpenAPI spec so the spec and the implementation can never silently drift.
/goal every path in openapi.yaml has a contract test asserting its status codes and response schema — add tests for one untested endpoint per turn, run the suite, and fix either the spec or the handler when they disagree (tell me which you chose); stop when all paths are covered or after 15 turns
claude-code · codex
Implementation note
The interesting output is the disagreements: each one is a real spec-vs-implementation bug. Make sure it reports which side it corrected.
More testing loops
Stabilize flaky tests for good
Measure the flakiness, fix one root cause at a time, and stop after a defined streak of stable full-suite runs.
Run [test suite] [N] times under the same conditions and list tests whose result changes. Fix the most frequent flake at its root cause—shared state, timing, ordering, or an external dependency—never with a blind sleep or retry. Run that test [N] times, then rerun the full suite. Repeat until [N] consecutive full-suite runs pass, progress stalls, or approval is required. Return each flake, root cause, fix, evidence, and justified quarantine.
Build a todo CLI app with tests
Ralph autonomously builds a todo CLI application with comprehensive test coverage over multiple iterations until complete.
/ralph-loop "Build a todo CLI app with tests" --max-iterations 30
Kill flaky tests
Run your test suite repeatedly, collect every intermittent failure, and fix or quarantine flaky tests until you get five consecutive green runs.
/loop run my test suite 20 times, collect every intermittent failure, fix or quarantine the flaky ones, and don't stop until you get 5 consecutive fully-green runs