Software needs decisions, not chat: testing TypeSafe's Jev inside Empryo

Every turn of an AI coding agent requires dozens of small judgment calls. Which skill should load for this request? Which search candidate is relevant? Did an API failure happen because of a temporary rate limit or an invalid parameter? Does this button click need user approval?

Today, agent harnesses either ask frontier models to answer these questions in conversational text—costing dollars and seconds—or rely on brittle text patterns.

On September 15, TypeSafe AI launched Jev, a dedicated "System One" decision model that gives up text generation completely. Instead of predicting tokens sequentially, Jev evaluates structured questions in parallel over an input state, returning calibrated choices in 70 to 300 milliseconds for $0.042 per million input tokens.

I wired Jev directly into Empryo across five decision points and tested eight potential jobs against real engineering sessions. Here is what I built, what I measured against frontier models, and what it taught me about where decision models belong in developer tooling.

What is a System One decision model?

Large language models predict the next token to hold a conversation. That design works for writing prose, explanations, and code. But software systems rarely need conversational text to take the next step. Software needs decisions: choose one option from a list, evaluate a true or false condition, or estimate confidence.

Jev does not chat. You supply an input state—such as a developer's request, a search candidate list, or an error payload—along with structured questions. Jev uses hardware-aware parallel sampling to answer all questions simultaneously, returning calibrated probabilities and confidence scores. Output tokens are free because the model outputs no text.

A search ranking request looks like this:

{
  "state": { "request": "find files calculating token cost" },
  "questions": {
    "target_file": {
      "type": "choice",
      "instructions": "Which file should the agent inspect first to find token cost calculations?",
      "criteria": {
        "billing/cost-alerts": "export function costAlerts(n)",
        "billing/cost-calc":   "export function costCalc(n)",
        "ui/cost-panel":       "export function CostPanel()"
      }
    }
  }
}

{
  "answers": {
    "target_file": {
      "choice": "billing/cost-calc",
      "confidence": 0.81,
      "probabilities": {
        "billing/cost-calc": 0.71,
        "ui/cost-panel": 0.17,
        "billing/cost-alerts": 0.12
      }
    }
  },
  "usage": { "input_tokens": 404, "output_tokens": 0 }
}

That evaluation completed in 266 milliseconds and cost $0.000017.

The five decision points inside Empryo

I integrated Jev into five distinct areas of Empryo where fast semantic evaluation helps the agent move quickly.

Every feature runs independently behind its own configuration switch and defaults to off. When disabled, Empryo operates using its standard deterministic logic.

The frontier killer: Computer use without burning frontier models

The received wisdom has been that driving a user interface requires a heavy frontier model like GPT-6 Astra or Claude on every step, taking screenshots and streaming tokens.

In practice, most interface actions are not complex reasoning problems. They are discrete choices over an indexed accessibility tree: which button to click, which menu to open, which field to focus. Jev evaluates those steps in 200 milliseconds with calibrated confidence scoring. If confidence falls below your set floor—or if a field requires free-form prose—the pilot immediately hands the wheel back to your frontier model.

Paired with Jev's action approval guard (which checks whether an upcoming click triggers payments, deletions, sends, or security changes), computer use gains frontier-grade safety and execution speed without burning dollars on simple navigation.

Measuring failure triage: Jev vs. frontier models

When an upstream model provider fails, the harness must decide whether to retry the request or halt the turn. Retrying a temporary rate limit saves a session; retrying a permanently malformed parameter wastes time and money on repeated rejections.

I evaluated Jev against frontier models across 102 real provider failures recorded in local session logs. The answer key was determined strictly by the provider's documented API specifications and error classifications.

Jev achieved 100% accuracy in 273 milliseconds, matching the accuracy of frontier reasoning models at one-fifth the latency and a fraction of the cost.

This benchmark also yielded an immediate architectural benefit: inspecting the four cases where Jev outperformed the original text pattern revealed that the status regex was matching the string 429 inside tool version identifiers (such as text_editor_20250429). Digit-bounding the status code in Empryo's core engine brought the free, built-in classifier to 102 of 102 as well. Using a calibrated decision model as an evaluation oracle helped harden the offline code.

Where deterministic code still wins

I tested eight potential jobs for Jev and discarded three after empirical measurement.

Here is why those three were discarded:

1. Ordering raw grep lines: Ranking individual text match lines with Jev dropped top-3 accuracy from 78.6% to 74.1%. A single line of code lacks the semantic context of a complete file or symbol, making lexical match position a more reliable signal.
2. Review loop pass/fail status: During automated self-correction loops, a model that misidentifies a failure report as a pass prematurely closes the run on broken code. Jev agreed with human reviewer classifications in 12 of 20 ambiguous cases, but false passes are unacceptable when verifying code correctness.
3. Predicting the next tool call: Simple keyword counting on the prompt outperformed both Jev (26% vs. 15%) and frontier models, because the intent to call specific tools is largely absent until initial file inspection occurs.

The takeaway is consistent: decision models excel when semantic interpretation of natural language is required and words do not match directly. When an exact index, graph traversal, or deterministic heuristic already captures the signal, deterministic code remains faster, cheaper, and more reliable.

Architecture: opt-in, circuit-broken, and transparent

I designed the TypeSafe integration to be resilient, completely opt-in, and transparent:

• 100% Opt-In: All features default to false. Users without a key experience no network calls, no behavior changes, and zero overhead.
• Circuit Breaker: If a key is revoked, invalid, or out of credits (HTTP 401 or 403), Empryo trips an automatic circuit breaker. Calls pause for 30 minutes and the system falls back instantly to standard heuristics. The harness never hangs or loops on a broken key.
• Transparent Ledger: Every Jev decision is logged with its exact input tokens, latency, and cost under its own model name in the usage ledger and displayed directly in the chat cue. It is never obscured or bundled into general token totals.

Enabling Jev in Empryo

To enable Jev, provide an API key and activate the specific decisions you want it to handle:

Summary

Generative LLMs are extraordinary tools for writing code and reasoning through complex software architectures. But asking them to make high-frequency micro-decisions inside an agent harness is slow and expensive.

System One decision models like Jev offer a compelling alternative: calibrated, typed decisions delivered in milliseconds at negligible cost. By deploying them selectively where semantic judgment matters—and keeping deterministic code where heuristics reign—coding agents become faster, cheaper, and more dependable.