Independent benchmark
important.is, a web agency in Poland, ran the same coding tasks with the same models in Empryo and in each model's own CLI: GLM 5.3 against Claude Code, Kimi against kimi CLI, GPT-6.1 against Codex CLI. A script checked every answer and a local proxy counted the tokens. Fewer input tokens in every matchup.
Run and published by
GLM 5.3, input tokens−51%
Same 19 tasks, 57 runs
Results at a glance
GLM 5.3: input tokens against Claude Code
−51%4.0M for Empryo against 8.2M for Claude Code on the same tasks. Output tokens dropped 52% too.
- GLM 5.3 tasks solved
- 19/19
- Claude Code solved 17 of 19 with the same model.
- GLM 5.3 full set
- 24 min
- Claude Code took 38 minutes for the same set.
- Kimi at max thinking
- −47%
- Input tokens against kimi CLI, 22% fewer output tokens, in similar time.
- Three models, three rival CLIs
- 113 runs
- Graded by a script, counted by a proxy, November 2026.
Fewer input tokens with every model
The rival CLI is set to 100. A shorter Empryo stroke means less text sent to the model for the same work.
GLM 5.3−51%
input tokens, 57 runs
Kimi, max thinking−47%
input tokens, 18 runs
Kimi, high thinking−28%
input tokens, 18 runs
GPT-6.1−17%
input tokens, 20 runs
Input is what the harness sends the model on every step: files, history and tool descriptions. Empryo keeps an index of the repository and picks the context from it, instead of sending more just in case.
Output tokens
Same scale, rival CLI at 100. With the codebase map in hand, the model has more to reason about, so on Kimi at high thinking it wrote a little more. The total still came out lower.
GLM 5.3−52%
output tokens, 57 runs
Kimi, max thinking−22%
output tokens, 18 runs
Kimi, high thinking+12%
output tokens, 18 runs
Still−28%input tokens
Slower on two, still fewer tokens on all three
Minutes for the whole task set. A shorter stroke is faster. Empryo hands the model a map of the codebase, which gives it more to think through on each step. That can take longer, but it still sends fewer tokens every time.
GLM 5.31.6× faster
minutes, whole set, 57 runs
And−51%input tokens
Kimi, high thinking1.7× slower
minutes, whole set, 18 runs
Still−28%input tokens
GPT-6.12.8× slower
minutes, whole set, 20 runs
Still−17%input tokens
Never fewer tasks solved
A script checked every answer. Same or better in all four matchups.
- GLM 5.3: Empryo 19/19, Claude Code 17/19
- Kimi, max thinking: Empryo 9/9, kimi CLI 9/9
- Kimi, high thinking: Empryo 9/9, kimi CLI 9/9
- GPT-6.1: Empryo 10/10, Codex CLI 10/10
They also found that Kimi's five-hour “requests” counter tracks usage, not calls: about 250 model calls moved it by about 50. Fewer tokens means more work inside the same plan.
The source
Written and measured by Łukasz Ślusarski at important.is, published 2 November 2026.
The write-up has the method, every table and the limits of the test. Read it on important.is