Custom providers

You can add any API that follows the OpenAI format as a provider. This includes hosted services like Together AI and servers you run yourself, like vLLM or llama.cpp. You only need a few lines of config. No code or plugin is needed.

Minimal exampleLink to this section

Add this to ~/.empryo/config.json (or .empryo/config.json to use it in one project only):

{
  "providers": [
    {
      "id": "together",
      "name": "Together AI",
      "baseURL": "https://api.together.xyz/v1",
      "envVar": "TOGETHER_API_KEY",
      "models": ["meta-llama/Llama-3-70b-chat-hf"]
    }
  ]
}

Save your key and use a model:

empryo --set-key together <your-key>
empryo --model together/meta-llama/Llama-3-70b-chat-hf

Model names are always <provider id>/<model id>.

Save the key from the command lineLink to this section

--set-key takes the provider's `id` from your config, not the environment variable name:

empryo --set-key together sk-...       # id from the config block
empryo --list-providers                # `together  ready  (TOGETHER_API_KEY)  [custom]`

Empryo stores the key in your system keychain, or in ~/.empryo/secrets.json if no keychain is available. It stores it under the envVar name you declared. So a provider needs `envVar` before you can save a key for it. Leave envVar out for a local server that needs no key. Include it for any service that does.

You can provide the same key in three ways:

If the provider is declared in a project's .empryo/config.json, run --set-key from inside that project, or pass --cwd <path>. Otherwise Empryo does not see the provider.

Trusting a projectLink to this section

Empryo ignores providers in a project's config until you trust that project. This protects you from a cloned repository that points your keys at a server you do not know. Until then, --set-key refuses the provider id even though you can see it in the file.

To trust the project from the same command:

empryo --set-key together sk-... --trust    # or --list-providers --trust

/trust in the terminal UI and the trust prompt in the desktop app do the same thing. Providers in your global ~/.empryo/config.json never need trust.

Several accounts on one providerLink to this section

providers is a list, and the same API can appear in it more than once. That is how you use two accounts on one service:

{
  "providers": [
    { "id": "together-work", "baseURL": "https://api.together.xyz/v1", "envVar": "TOGETHER_WORK_KEY", "models": ["meta-llama/Llama-3-70b"] },
    { "id": "together-personal", "baseURL": "https://api.together.xyz/v1", "envVar": "TOGETHER_PERSONAL_KEY", "models": ["meta-llama/Llama-3-70b"] }
  ]
}

You get two providers with two keys, and you choose which one a tab uses by picking its model (together-work/meta-llama/Llama-3-70b).

If you want Empryo to switch accounts automatically, keep one provider and save both keys for it. Running --set-key a second time adds the new key and keeps the first one:

empryo --set-key together sk-account-one
empryo --set-key together sk-account-two

When one account runs out of quota during a turn, Empryo moves to the next saved key. When every key is used up, it stops and shows you the provider's error.

A short rate limit is handled differently. Empryo waits and retries with the same key, instead of moving to your other accounts for a limit that clears in seconds. A key that comes from an environment variable is never swapped out.

FieldsLink to this section

Servers that reject JSON SchemaLink to this section

Some Empryo features, such as code review and planning, ask the model for an answer in a fixed format. They send that format as response_format: {"type": "json_schema", ...}. Most OpenAI-compatible servers accept this. Some older vLLM and llama.cpp versions, and some self-hosted gateways, only accept {"type": "json_object"}.

For those servers, set structuredOutputs: false:

{
  "providers": [
    {
      "id": "homelab",
      "baseURL": "http://192.168.1.40:8000/v1",
      "structuredOutputs": false
    }
  ]
}

Empryo then sends json_object and describes the format in its instructions to the model. This is less reliable, but it works. Only change this setting if your server rejects a request and mentions response_format.

Declare model limitsLink to this section

Empryo cannot tell how large a self-hosted model's context window is. If you do not declare it, Empryo assumes 128k tokens. If your model is smaller, the server rejects requests long before Empryo decides to compact the conversation, and it looks as if compaction is broken.

Declare the real limits for each model:

{
  "providers": [{
    "id": "local",
    "baseURL": "http://localhost:8080/v1",
    "models": [
      { "id": "qwen-32k", "name": "Qwen 32k", "contextWindow": 32768, "maxOutputTokens": 4096 }
    ]
  }]
}

• contextWindow sets the size of the context meter and when automatic compaction starts.
• maxOutputTokens sets how much room Empryo keeps for the model's reply.
• If your server's /v1/models endpoint reports a window size, Empryo uses that value. Your declared value is the fallback.
• To override one model without editing the provider, add "contextWindowOverrides": { "local/qwen-32k": 32768 } at the top level of your config.

More examplesLink to this section

{
  "providers": [{
    "id": "local",
    "name": "Local LLM",
    "baseURL": "http://localhost:8080/v1",
    "models": ["llama-3-70b"]
  }]
}

{
  "providers": [{
    "id": "corp",
    "name": "Corp Gateway",
    "baseURL": "https://llm.internal.corp.com/v1",
    "envVar": "CORP_LLM_KEY",
    "modelsAPI": "https://llm.internal.corp.com/v1/models"
  }]
}

{
  "providers": [
    { "id": "together", "baseURL": "https://api.together.xyz/v1", "envVar": "TOGETHER_API_KEY", "models": ["meta-llama/Llama-3-70b"] },
    { "id": "cerebras", "baseURL": "https://api.cerebras.ai/v1", "envVar": "CEREBRAS_API_KEY", "modelsAPI": "https://api.cerebras.ai/v1/models" }
  ]
}

Thinking and reasoningLink to this section

Some models can think before they answer. To turn that on over an OpenAI-compatible API, add a reasoning block. Providers expect different fields, so use the style yours accepts:

{
  "providers": [{
    "id": "dashscope",
    "baseURL": "https://dashscope.aliyuncs.com/compatible-mode/v1",
    "envVar": "DASHSCOPE_API_KEY",
    "models": ["qwen-plus"],
    "reasoning": {
      "enabled": true,
      "budget": 8192
    }
  }]
}

Empryo adds these fields to every request. If a field in extraParams has the same name as another field, extraParams wins.

auto - let Empryo pick the bodyLink to this section

Without a reasoning block, Empryo does not ask your API for any thinking. A reasoning model then thinks at whatever level the server defaults to, and you may never see it.

If you do not know which fields your server expects, turn on auto:

{
  "providers": [{
    "id": "my-gateway",
    "baseURL": "https://llm.internal.corp.com/v1",
    "envVar": "MY_GATEWAY_KEY",
    "reasoning": { "auto": true }
  }]
}

Empryo then looks up each model in the models.dev catalog. It sends the standard OpenAI-style reasoning setting only for models the catalog lists as reasoning models, and only at effort levels the catalog says the model supports. Where the vendor has its own on/off switch, Empryo sends that too (enable_thinking for Qwen, thinking: { type } for GLM and Kimi). Models the catalog does not know, or lists as non-reasoning, get no extra fields.

auto is off by default. Providers without it behave exactly as before. If you also set effort, enabled, budget, or extraParams, Empryo uses those and ignores auto. /effort and --effort still work either way. An effort level you choose beats the automatic one.

Try auto when your provider serves a reasoning model and you never see any thinking.

Which effort levels a model offersLink to this section

The effort control (/effort, Alt+E in the terminal, the desktop effort menu and its shortcut) shows the levels the models.dev catalog lists for a model. A model the catalog does not know has no levels to show. Declare them yourself with efforts, per model or for the whole provider:

{
  "providers": [{
    "id": "my-proxy",
    "baseURL": "https://llm.internal.corp.com/v1",
    "envVar": "MY_PROXY_KEY",
    "models": [
      { "id": "qwen3-max", "name": "Qwen3 Max", "efforts": ["low", "high"] },
      "deepseek-v4"
    ],
    "reasoning": { "efforts": ["low", "medium", "high"] }
  }]
}

A model's own efforts wins. Models without one use reasoning.efforts. Allowed levels are none, minimal, low, medium, high, xhigh and max. Off is always offered. If a level you picked earlier is not in the new list, Empryo sends the nearest one the model has.

If Empryo can't use a list, for example "efforts": "high" instead of ["high"], a level it doesn't know, an empty list or a level listed twice, it names the list and where it is, such as providers[0].models[0].efforts, and ignores it. The model then uses reasoning.efforts or the default levels.

How effort is sentLink to this section

By default, effort is sent as the flat OpenAI field, { "reasoning_effort": "high" }, which every OpenAI-compatible API understands. If your server expects the nested OpenRouter format, set effortStyle:

{
  "reasoning": {
    "effort": "high",
    "effortStyle": "nested"
  }
}

Only use "both" if your server really needs it. Strict servers reject a request that contains both fields, with an error like *"Pass either reasoning (nested object) or reasoning_effort (flat field), not both."*

Extra request-body parametersLink to this section

Some APIs turn on a feature through a field in the request, not through a tool. DashScope reads enable_search, Zhipu reads web_search, and your own gateway may read whatever you built it to read. extraBody adds the JSON you write to every chat request sent to that provider.

{
  "providers": [{
    "id": "dashscope",
    "baseURL": "https://dashscope.aliyuncs.com/compatible-mode/v1",
    "envVar": "DASHSCOPE_API_KEY",
    "models": ["qwen-plus"],
    "extraBody": { "enable_search": true }
  }]
}

The desktop app has the same field in its provider dialog, below Models. It checks your JSON as you type and only lets you save valid JSON.

Empryo adds these fields last, so they replace any field with the same name. If your API rejects a field, the whole request fails. Empryo does not silently drop it.

reasoning.extraParams is different. It is meant for thinking settings, so Empryo stops sending it when effort is off. extraBody is sent at every effort level. Put thinking settings in reasoning.extraParams and everything else in extraBody.

To add fields for a built-in provider instead of a custom one, use the `extraBody` setting at the top level of your config, keyed by provider or model id.

Turn a provider off without deleting itLink to this section

To pause a provider, set "disabled": true on it, or click its power button in the desktop app's custom provider dialog. Empryo keeps its config, key, and model list, but hides its models and stops contacting it. It stays in the provider list, grayed out, so you can turn it back on with one click.

This is handy when a local server is offline and you do not want its models in the picker, or when you switch between two endpoints and want to keep the spare one.

ScopeLink to this section

• Global: ~/.empryo/config.json applies to every project.
• Project: .empryo/config.json applies to one project. A project entry replaces a global entry with the same id.

Name conflictsLink to this section

If your id matches a built-in provider (for example "anthropic"), Empryo renames yours to {id}-custom. It never replaces the built-in one. Custom providers are always marked [custom] in /keys and --list-providers.