Benchmarks
Code has a shape
Most coding agents read your repo as one long wall of text. Empryo learns the shape of it: who calls what, which files move together. We bet real money that the shape wins. Four real bugs, six models, every dollar measured. Here is the tab.
The smarter the model, the cheaper Empryo gets. Smart models know what to do with a map. Skipping the map only pays where tokens are nearly free.
* On Haiku a one-cent helper model preps the ground first. pi's one miss was a stall of its own. pi ran at its barest; Empryo carried everything.
One side packed light. We brought the map.
pi at its cheapest possible shape. Empryo at full weight.Every real pi setup grows past this shape. Each tool you add makes every step read more. Empryo carried 37 tools into this fight and still won where models cost real money.
Our own engine learned this the hard way
Forge v1 → Forge v2 · same bugs, same models, same keys · cost bars: lower = cheaperForge v1 showed every model the whole map on every step, and paid for it. Forge v2 asks who is looking. Strong models get the short tour. Small models get the full manual. Tools lose the right to dump a book into the chat. The bill dropped by half.
Where models cost real money, the map wins
Opus 5 · cost per task, all three fixed everything · lower bar = cheaperAn early v2 build lost this tier. One search result the size of a small book got re-read on every later step. v2 now caps what a tool may say. That one change beat pi by 27% on cost and 40% on time, with 60 requests against 80.
faster to the right file. Two files in the big repo share one name; only one holds the bug. An agent reading strings has to guess. Empryo followed the map and knew: 86 seconds. pi searched for 369. Our old engine guessed wrong and patched the other file.
Stop treating code as strings.
Functions call functions. Files move together. Hand your agent that map and every dollar works harder.
The fine print
Most numbers average two runs; a few are single runs. Both test repos are TypeScript. The bench wipes git history before each run, which blinds one of Empryo's senses. If anything these numbers undersell it. Don't take our word for it — the whole cage (tasks, hidden tests, metering) lives in the public bench repo.