Walk away, read it over coffee
Part of my EVA series, where I build the cockpit I pilot GORED runs from. This is the last one, and it is the mode the whole cockpit was pointed at: the overnight blind campaign. Code in t0ul/GORED; write-ups in t0ul/ai-security-engineering.
Intro
Here is the honest advantage of a tester that runs on my own laptop. It is not that it is smarter than a cloud model, because my GORED series is clear that it is not. It is that it costs nothing per hour and never gets bored. A frontier model billed by the token is a thing you ration. A small model running locally is a thing you can leave on all night, pointed at a problem, for the price of some electricity.
So the mode this whole cockpit was building toward is the one where I stop piloting entirely. I name a few of my own apps, turn the autonomy dial all the way up, click launch, and go to bed. The campaign runs them one after another, pass after pass, until I stop it or the morning comes. And over coffee there is a report of everything it tried, everything it thinks it found, and how those findings might chain together. This post is how that mode works, and the one thing that makes it trustworthy against apps where nothing is planted.
I. Blind means every finding stands on its own
The lab targets I used to build GORED had planted ground truth: I knew what was wrong with them, so a capture could be checked against an answer key. My own live apps do not work like that. Nothing is planted in RiskRancher or emaildrop or krabber, which means there is no answer key, which means a finding cannot lean on one. It has to carry its own evidence.
That is what the confidence ladder is for, and it is why I built it before I built this. A blind finding is only as good as the self-evidencing signal underneath it, so the validator ranks each one. A cross-site scripting hit that a scanner actually fired and confirmed on the real response is near-certain. A mass-assignment write that shows up on an independent read-back is certain, because the app itself is now telling you the change stuck. A forged token the server accepts on replay is certain. A broken-access-control case, where a low-privilege identity can read a privileged identity’s distinctive data, is scored by how distinctive that data is, with no planted canary needed. A system-prompt leak with no known secret to match is honestly marked lower, because in blind mode I cannot be sure. Nothing here needs a flag. Each signal is the app betraying itself on its own real bytes, which is the only kind of finding I trust from an unattended run.
II. Non-stop, by design
A campaign is a named set of targets run autonomously, and autonomous here means exactly what it meant two posts ago: no checkpoint installed, the dial all the way up, byte-for-byte the headless loop. The durable record of it is deliberately thin, an envelope around the child runs:
// Campaign is the envelope around an overnight blind campaign. The per-target detail
// lives in the child eva_runs and their traces; this survives a restart so the morning
// report can find its runs.
type Campaign struct {
ID string
Targets string // the named targets
Model string
Status string
RunIDs string // the child runs this campaign spawned
Cur string // the "target · stage" in flight right now, for live status
Loop bool // keep re-running the targets in passes until stopped: the overnight mode
Pass int // how many passes the loop has started
}
The Loop flag is the whole overnight idea. With it set, the scheduler does not run each target once and stop; it runs the full pipeline against every target, then starts another pass, and another, until I abort it or it runs out of night. Models are used one at a time, because two of them will not fit in my laptop’s memory at once, so the campaign serializes them. And because the campaign and its child runs are all written to the same durable registry from the last post, the thing survives me closing the lid and the server restarting. I can wake up, reopen the cockpit, and the campaign is still there with its passes counted.
III. It remembers across passes
A night of passes would be pointless if each pass started from zero, rediscovering the same surface every time. So the campaign has a memory that accumulates. Every endpoint a pass discovers is written to a project surface for that target, with what the recon learned about it. Every finding is written to a project findings table, attributed to the target and the model that found it. The next pass reads those first, so it does not re-map what pass one already mapped; it picks up where the night left off.
The piece I am fondest of is the exploit frontier. When a pass works an endpoint, it records that it tried, and whether something landed:
func (s *Store) MarkAttacked(target, method, path string, found bool) error
So a later pass can see that an endpoint has already been worked and move on to fresher ground, instead of hammering the same door all night. Contracts about an endpoint, what fields it takes, whether it needs auth, merge across passes and keep the best-provenance version. The effect is that eight hours of passes behave less like eight hours of the same hour and more like a search that genuinely widens. Findings discovered hours apart can then be linked into a chain, because they all live in one accumulating project memory rather than evaporating with each run.
IV. The report you read over coffee
The artifact at the end is a per-target report. For each app it lays out coverage, the stages, tools, paths and vulnerability classes it attempted, with counts, so I can see not just what it found but what it looked at and came up empty on, which is often the more useful half. Then the findings, ranked by that confidence ladder, each with its class and a repro I can paste into a terminal to see for myself. Then any chains, findings linked into an escalation. And a link back to each pass’s trace, so every claim in the report drops straight into the hash-verified record that backs it. A campaign overview rolls the targets up into one page.
I want to be plain about the safety of this, because “point an autonomous attacker at your apps and sleep” deserves a straight answer. These are my own applications; I am authorized to test them. The discipline is not about permission, it is about not breaking my own things while I am not watching. Every write a probe makes is self-scoped and reversible by design: it edits only the testing identity’s own records, never deletes, never mass-mutates, never touches another user’s data. Containment still holds, with the egress allowlist scoped to each target, and the per-run bounds still apply. The point is to wake up to a report, not to a corrupted database.
Conclusion
That is the cockpit, finished enough to fly. EVA took a pile of command-line sweeps I could only grep and turned them into something I can launch from a form, watch think in real time, trust the recording of because the chain cannot be quietly rewritten, compare and rank, and now walk away from entirely and read in the morning. It did not make the small models better at finding bugs, and I have been honest in the other series about where they top out. What it did was make the work I could do with them something I could actually see, reason about, and leave running.
If you have followed both series, the shape is probably clear now. The science said a prompted local model mostly cannot find your bugs, and the honest next step lives on a GPU. The engineering said the instrument around it is still worth building, because recon works today and an unattended campaign over your own apps costs nothing but a night. The weapon is not sharp yet. The cockpit is real. I would rather ship the real cockpit and tell you the truth about the weapon than the other way around.