Agent Loop Safety: Hooks, Budgets, and Evaluators

A loop running claude -p with --dangerously-skip-permissions has no human between the agent and your filesystem. Something mechanical has to do the review a human is not there to do. Three controls cover it: hooks veto bad commands before they execute, budgets cap what a bad run can cost, and evaluators reject bad results before the harness accepts them. Isolation sits under all three. Working scripts for each below.

The threat model

An agent optimizing for its exit condition takes any shortcut you did not forbid. Almost none of the failures are malice. Five recur:

Each layer below kills a different subset.

Layer 0: Isolation — decide the blast radius first

Before hooks or budgets, decide what a worst-case iteration can destroy:

# Worktree: cheap, good default for code loops
git worktree add ../loop-run -b loop/attempt-1
cd ../loop-run && claude -p "$(cat prompt.md)" --dangerously-skip-permissions

Layer 1: Hooks — block bad actions in real time

Claude Code hooks run your code on the agent's lifecycle events. A PreToolUse hook fires before every tool call and can veto it. The harness enforces the veto; the agent cannot argue with an exit code.

.claude/settings.json:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [{ "type": "command", "command": "./hooks/deny-dangerous.sh" }]
      }
    ]
  }
}

hooks/deny-dangerous.sh — exit code 2 blocks the call and tells the agent why:

#!/usr/bin/env bash
cmd=$(jq -r '.tool_input.command')
deny='rm -rf|git push --force|git push -f|curl[^|]*\|\s*(ba)?sh|git reset --hard|chmod -R 777|> \.env'
if echo "$cmd" | grep -qE "$deny"; then
  echo "Blocked by policy: $cmd" >&2
  exit 2
fi
exit 0

The same denylist matters when you publish loop content, not just when you run it. Our ingest pipeline default-rejects any submitted loop containing rm -rf, curl | sh, force-pushes, or secret-reading commands, for exactly this reason.

Two more hooks belong in every harness:

• PostToolUse on Edit/Write — run the formatter and a fast lint on any file the agent touches, so drift is corrected the moment it happens.
• Stop hook — verify the agent wrote its handoff notes before the session ends. ralph-guardrails-learning builds on this pattern, appending learnings every iteration.

Layer 2: Budgets — bound the cost of being wrong

The harness enforces budgets. Never the model — the model is the thing being budgeted.

MAX_ITER=15
DEADLINE=$(( $(date +%s) + 3600 ))   # 1 hour wall clock

for i in $(seq 1 "$MAX_ITER"); do [ "$(date +%s)" -gt "$DEADLINE" ] && { echo "budget: time"; break; }

timeout 15m claude -p "$(cat prompt.md)" --dangerously-skip-permissions

if npm test && npm run lint; then echo "goal met on iteration $i"; exit 0 fi done echo "budget exhausted — leaving notes in progress.md for human review" exit 1 ```

Budget every axis independently: iterations (the outer loop), wall clock (the deadline check), per-iteration time (timeout 15m, so one hung command cannot eat the whole budget), and spend, via your API console's limits. A stuck loop should end as a report in progress.md, not as a bill.

This is the layer people skip, and we can measure it. Every loop in this directory is scored on six prompt signals:

Across the published corpus, the two budget signals — a stated exit condition and an iteration cap — are consistently the rarest. They are also decisive by construction: the arithmetic above makes an A impossible without both, and a loop carrying either one cannot land at D.

One addition pays for itself fast: a no-progress detector. If the diff between iterations is empty twice in a row, the loop is spinning — stop early and page a human. That is the moment to send an email, and giving agents a clean way to send one is what ConnectMyEmail is for.

Layer 3: Evaluators — judge the result before accepting it

Hooks catch bad actions; evaluators catch bad outcomes. Before the harness accepts an iteration's work, an independent check answers one question: did the loop achieve the goal legitimately?

The rule: evaluators are default-fail. If a check cannot verify the work, it rejects the work.

evaluate() {
  # 1. Goal actually met
  npm test && npm run lint || return 1

# 2. No test attrition — the classic reward hack before=$(git show origin/main:test-manifest.txt | wc -l) after=$(npm test -- --listTests | wc -l) [ "$after" -ge "$before" ] || { echo "FAIL: test count dropped"; return 1; }

# 3. Diff stayed in scope git diff --name-only origin/main | grep -vE '^(src|tests)/' && \ { echo "FAIL: out-of-scope files touched"; return 1; }

# 4. No suppressions smuggled in git diff origin/main | grep -E '\.skip\(|@ts-ignore|eslint-disable' && \ { echo "FAIL: suppression added"; return 1; }

return 0 } ```

For fuzzier goals, add a second-model review: a separate claude -p call with read-only access that grades the diff against the original intent and returns PASS/FAIL. Keep it separate from the worker — a model grading its own diff passes it. This is the check behind kill-flaky-tests demanding ten consecutive green runs instead of one, and behind reach-coverage-target confining the agent to tests/.

The stack, assembled

Every layer here is a few lines of shell. The loop builder generates the whole stack — hooks file, budgeted harness, default-fail evaluator — from your goal and scope, so the bounds exist before the loop runs its first iteration.