Observability Is Part of the Harness, Not an Afterthought

The usual order of operations is: build the loop, run it overnight, and add logging the morning you find it did something wrong. By then it's too late. The terminal has closed, the scrollback is gone, and the one question you actually need answered — what did it do, and why did it stop? — has no evidence behind it. You can't debug what you didn't record.

That's the argument the Learn Harness Engineering course makes in its lecture on why observability belongs inside the harness, and it's the right one. The course is a genuinely good complement to this site — it treats the harness as an engineering discipline rather than a prompt trick, and this is one of the places it earns that framing. We arrived at the same conclusion from the other direction: we grade loops, and the ones that go quiet are the ones we can never salvage.

Silent success is worse than silent failure

A loud failure is a gift. The run crashes, you get a stack trace, you fix it. The dangerous mode is the quiet one — and it has two shapes, not one.

The first is silent failure: the loop stops, does nothing useful, and leaves no trace of where it gave up. Annoying, but recoverable — you rerun it.

The second is worse: silent success. The loop reports done, exits clean, and did the wrong thing. It refactored the wrong module, "fixed" a test by deleting the assertion, committed a change that passes CI and breaks in production. Nothing looks broken. You find out days later, and by then the only record of what happened is a git log entry that says chore: updates. There's no way to answer did it actually do the thing? because nothing recorded what "the thing" was supposed to be or what the verifier saw when it waved the run through.

Observability is what turns both of those from unrecoverable into diagnosable.

What minimal observability actually is

Not a dashboard. Not distributed tracing. For an unattended loop, the floor is a single durable record per run that answers three questions an owner can read cold, without having watched it live:

What was attempted — the goal string, the iteration count, the diffs or actions taken.
What the verifier said — pass or fail, and the specific reason. Not "checked," the actual verdict and evidence.
Why it stopped — hit the cap, passed the gate, hit an error, or escalated.

The key word is durable. Stdout scrollback is not observability; it vanishes the instant the terminal closes, which for an overnight loop is exactly when you weren't watching. The record has to outlive the run — a per-run log file, an appended state entry, a beacon posted to an endpoint. Something you can open on Tuesday to explain what happened Sunday night, with enough in it that "how do I know?" has an answer other than a shrug.

The harness fixes this structurally, not behaviorally

This is the same house argument we make about safety, and it's worth stating plainly: you don't get observability by asking the model to log carefully. It won't, and even when it does, self-reported logs are as trustworthy as a maker grading its own work — which is to say, not. You get observability by building a pipeline where every run emits its record as a side effect of stopping, whether the agent cooperates or not. The trigger writes the start; the verifier writes its verdict; the exit writes the reason. The model's diligence never enters into it.

That's the sibling relationship between the two primitives. Guardrails stop the bad action before it happens — the deny list, the cap, the kill switch. Observability tells you what happened after — the durable account you read when you want to know whether the guardrails ever fired, or whether the loop sailed through and did something subtly wrong inside its permissions. One is prevention, one is forensics. A loop with guardrails and no observability can't be trusted unattended, because the first time it misbehaves inside its allowed boundary, you'll have no idea it did.

Sites in this space, including this one, wire this in as run reporting: each loop posts a beacon on run_end with what it did, what the gate returned, and why it halted — a record that exists before anyone asks for it. That's the pattern, whatever you call it.

It's a grading dimension, not a nice-to-have

We don't treat this as optional, and neither should you. Loops in the directory are graded A–F partly on observability — whether a failure is visible and loggable, not merely whether a stop condition exists. A loop can have a perfectly good cap and still floor, because a bound that halts the run silently tells you that it stopped, never what it left behind. A stop condition without a record is half a safety feature.

So build the log with the loop, on run one, before you ever walk away from it. It costs a few lines and it's the difference between a loop you can diagnose and a loop you can only guess about. When the quiet failure comes — and on a long enough timeline it comes — the owner who recorded what happened fixes it in an afternoon, and the one who didn't starts over from nothing.

If you want the failures this is protecting you from, Loops Gone Wrong is the casebook, and How to Keep an Agent Loop From Running Away is the companion on the stop conditions that observability makes legible.

Observability Is Part of the Harness, Not an Afterthought