Two things got past me that no test suite I could write would have caught.
The first was an installer. A fresh Debian guest, no Empryo on it, no Bun, and no TERM variable, because nothing sets one when you exec into a container.
The script did everything right and then ran clear to tidy the screen before printing its welcome banner. clear needs TERM. It exited 1, set -e did the rest, and the script died holding a verified binary it never unpacked.
The second was worse, because nothing failed at all. The desktop app shipped with no right-click menu in any editable field. You could not paste with a mouse. Every test stayed green, because no assertion existed to go red. A user found it immediately.
Those are two different failures and they need two different answers. The first got through because my tests import my code, so they had already skipped the install. The second got through because a test can only ever re-ask a question somebody already thought to ask.
Full disclosure as always: I build Empryo alone, nights and weekends, unfunded. Everything below comes from the repo, and most of the numbers regenerate from a command you can run.
Two operators, one system
So I built one system and gave it two ways to be driven.
I run it deterministically. bun run qa executes 260 checks across three operating systems and three ways of running Empryo, and hands back the same answer every time. This half re-asks questions. It is the half I trust.
An agent drives it exploratorily. Same seeds, same machines, same reviewers, except the agent chooses what to attack, drives the product the way a person would, and writes the check it wishes had existed. This half finds new things. It is the half I check.
Neither half may promote its own findings, which turns out to be the part that makes the rest usable.
What the deterministic half drives
A seed declares what must hold: what to exercise, on which operating systems, in which of the three ways you can run Empryo, and one sentence naming the bug class it guards. A killer cell binds one seed to one platform and one surface and asserts. A patrol is one run, sweeping everything reachable in parallel.
Nothing is faked except the model. A real pty, a real Electron window, a real Linux guest under Apple's container runtime, and a real SSH session to a real Windows x64 box on my own network. The LLM is scripted through an OpenAI-compatible server, so the agent loop and the tool dispatch and the wire format all run for real, offline and free.
The Windows target has to be a real x64 machine. I own an ARM64 Windows VM and it is only allowed to smoke-test, because it cannot load x64 native modules, so a green run there would be green for a product nobody ships.
The part I did not expect to earn its keep is the innate sweep, which fires on every single result whatever the seed was looking for: stack traces, escape codes printed as literal text, SQLITE_BUSY, a process somebody had to kill. Most surprises arrive there. A seed about resuming an unknown session is how I found a segfault in Bun's HTTP client thread, which had nothing to do with sessions and was killing about 40% of runs.
How an agent picks what to attack
Empryo's own code intelligence turned out to be a QA tool.
An agent has to choose a target. With grep it picks whatever it can spell, which means the file whose name sounds most like the feature. That file is rarely the one that matters.
The graph answers a better question. genome_impact returns the dependents, the co-changes and the blast radius of a file. genome_analyze finds the exports nothing calls and ranks identifiers by how often the repo references them. navigate returns every caller that can reach a given function, as file:line.
That last one does double duty, and I only noticed after wiring it up. The set of callers that reach a function is also the set of surfaces and platforms a defect in it can reach, which is exactly what a seed has to declare. The tool that finds the bug writes half the seed's header.
Six agents share this system, and one detail about them matters more than the rest: none of them can call another. They all declare the same six tools and not one has a delegation tool, so every handoff in their instructions means the agent writes down what it found and a person starts the next one. That constraint stays. An agent that could promote its own finding straight into the suite would fill it with assertions that fire on correct behaviour.
Nothing becomes a check until something tries to kill it
Hunters want to find things. Left alone they produce a suite full of assertions that fire on correct behaviour, and then a red patrol stops meaning anything at all.
So every claim faces two reviewers before it can become a seed. One asks whether the behaviour is the product's documented intent. The other ignores the harness entirely and reproduces by hand on the shipped build, in a throwaway directory, the way a user would type it. The default verdict is refuted.
One pass came back with 23 red seeds on Windows. Five were real. Sixteen traced to a single stale binary the guest had been pinned to, and two to seeds somebody wrote POSIX-shaped. Killing eighteen false findings is the most valuable thing that filter does. Refuting is a win here, not a failure.
A QA writeup where nothing is ever refuted is measuring its reviewers, not its findings. So the ledger is append-only and publishes the refutations beside the confirmations: 105 claimed, 84 confirmed, 19 refuted, 2 disputed.
What it caught
Three defects, fixed now, each leaving a permanent check behind:
| what a user was losing | |
|---|---|
edit_file wrote through a read-only file | You chmod 444 a generated file. The agent edited it anyway and was told the edit succeeded, then kept reasoning about a file it should never have touched. |
| Editing a file lost its permissions | The atomic write renamed over the target, so the mode became the umask default. Every edited script quietly stopped being executable. |
--resume on an unreadable session answered with no history | The session file was unreadable, so the agent started empty, answered confidently, and billed you for a turn that forgot the conversation. |
None of those threw. None exited non-zero. All three looked exactly like success from the outside, which is why they survived a unit suite.
The day-one seeds, the ones that install Empryo on a bare machine and watch what a stranger sees, found four more: the clear death above, a tar: warning printed once per file because macOS packed extended attributes into the tarball, a warning claiming Empryo needs Bun installed when it ships its own, and curl's progress bar smearing into piped logs.
Another one: a release build failed its own smoke gate with exit null. The binary was fine. boot.tsx swept for orphaned language servers before reaching the --version fast path, that sweep touched ~/.local/share, and on a machine with legacy data it triggered a migration that outran the gate's timeout. Ten lines of stack trace, one moved call.
What is not covered
Empryo exposes 373 reachable capabilities. 96 have a seed. That leaves 277 a patrol would sail straight past, and the file saying so regenerates from the code, so I cannot quietly stop counting the ones I have not done.
Of the eight things a person does with a tool, from installing it to giving it up, three are proven end to end on a machine that has never seen Empryo. The other five are not. A journey only counts when it starts from a bare machine. Arranging the world first and then declaring it covered is the easiest lie to tell yourself here, so the registry refuses to accept one.
The thing I have not built
I want to be precise, because this is the sentence people skip to.
This system does not fix itself. Agents find defects and write the checks that guard them, and they are good at it. But every patrol is a command someone types, every finding is relayed by hand to the reviewer that attacks it, and every fix is still a person deciding what the right behaviour is. Nothing runs unattended. Calling it self-healing would be exactly the kind of claim this whole apparatus exists to catch.
What is true is narrower and more interesting: the loop closes. A defect gets found, argued about by something whose default answer is no, fixed, and then guarded by a check born from its own corpse. Every turn leaves the suite bigger and the gap smaller. 84 confirmed defects across three adversarial passes is a good start and also a slightly alarming statement about what was in there.
What would have to be true for the rest? Scheduled patrols, so a run is not a thing I remember to do. Agents that hand off to each other under a policy instead of through me. And a fix loop that proposes the patch and proves it against the check that failed, then stops, because the last call should stay human.
Point it at your own app
None of this is specific to Empryo. It is specific to having a product and wanting to know whether it works.
The pieces are general. Drive the real artifact instead of importing it. Read the code graph to choose targets instead of guessing. Refuse to let a claim become a test until something hostile fails to kill it. Publish the gap next to the coverage. What is Empryo-shaped is that the code intelligence is already in the box, so the thing doing the auditing can read the thing being audited.
I keep looking at that and thinking it should not stay in my repo. 👀
The system is at empryo.com/immunity, regenerated on every deploy, gaps and all.