# Custom providers > Connect any OpenAI-compatible API to Empryo with one config block: set baseURL, key env var, models, dynamic model lists, and reasoning options. Add any OpenAI-compatible API as a provider. No code, no plugin. ## Minimal example Add this to `~/.empryo/config.json` (or `.empryo/config.json` for project scope): ```json { "providers": [ { "id": "together", "name": "Together AI", "baseURL": "https://api.together.xyz/v1", "envVar": "TOGETHER_API_KEY", "models": ["meta-llama/Llama-3-70b-chat-hf"] } ] } ``` Set the key and use it: ```bash empryo --set-key together empryo --model together/meta-llama/Llama-3-70b-chat-hf ``` ## Set the key from the CLI `--set-key` takes the provider **`id`** from your config, not the env var: ```bash empryo --set-key together sk-... # id from the config block empryo --list-providers # `together ready (TOGETHER_API_KEY) [custom]` ``` The key is stored in your system keychain (`~/.empryo/secrets.json` when no keychain is available) under the `envVar` name the provider declares, so **`envVar` is what makes a provider keyable**. Without it there is nothing to store the key under and `--set-key` refuses the provider, that is the right shape for a local server that wants no auth, and the wrong one for a gateway that does. Three ways to hand over the same key, in the order they win: | Where | How | Wins when | |---|---|---| | Environment | `export TOGETHER_API_KEY=sk-...` | always, unless `keyPriority: "app"` | | CLI | `empryo --set-key together sk-...` | no env var set | | UI | `/keys` on the TUI, Settings → Providers on desktop | no env var set | Config is loaded before the key is saved, so a provider declared in the project `.empryo/config.json` needs `--cwd` (or a shell already in that repo) for `--set-key` to find it. An **untrusted checkout drops `providers` entirely**, a project-scoped provider is then invisible and `--set-key` refuses an id you can plainly read in your own config. Grant trust once, from the same command: ```bash empryo --set-key together sk-... --trust # or --list-providers --trust ``` `/trust` inside the TUI and the desktop trust prompt do the same thing. Global providers in `~/.empryo/config.json` are never gated. ## Several accounts on one provider `providers` is an array, and the same upstream can appear in it more than once. That is how you run two accounts against one API: ```json { "providers": [ { "id": "together-work", "baseURL": "https://api.together.xyz/v1", "envVar": "TOGETHER_WORK_KEY", "models": ["meta-llama/Llama-3-70b"] }, { "id": "together-personal", "baseURL": "https://api.together.xyz/v1", "envVar": "TOGETHER_PERSONAL_KEY", "models": ["meta-llama/Llama-3-70b"] } ] } ``` Two ids, two keys, two rows in the picker, you choose which one a tab bills to (`together-work/meta-llama/Llama-3-70b`). For **automatic** hand-over, keep one entry and store both keys under it: run `--set-key` again with the second key and the first is kept beside it rather than replaced. ```bash empryo --set-key together sk-account-one empryo --set-key together sk-account-two ``` Empryo moves to the next stored key when one account runs out of quota mid-turn, and stops when every key is spent. You then see the provider's own refusal. A short rate limit is different: Empryo waits on the same key rather than spending your other accounts on a throttle that clears in seconds. A key that comes from an environment variable is never rotated, because it is not Empryo's to swap. ## Fields | Field | Required | What it does | |-------|----------|--------------| | `id` | yes | Used in model strings (`together/llama-3`) | | `baseURL` | yes | OpenAI-compatible endpoint | | `name` | - | Display name | | `envVar` | - | Env var for the key (omit for auth-less servers) | | `models` | - | Model list (strings or `{id, name, contextWindow, maxOutputTokens}`), declared limits drive the context gauge and auto-compaction. **Declaring any turns `/models` discovery off** | | `modelsAPI` | - | URL returning OpenAI `/v1/models` format. Set it beside `models` to discover *and* keep your ids; `false` disables discovery outright | | `disabled` | - | Park the provider: models leave the picker, nothing fetches its endpoints | | `reasoning` | - | Thinking/reasoning config, including `auto` (see below) | | `extraBody` | - | JSON merged into every chat request to this provider (see below) | | `structuredOutputs` | - | Default `true`. Set `false` for an endpoint that rejects a `json_schema` response format (see below) | ## Endpoints that reject a JSON Schema Some parts of Empryo ask a model for a fixed shape of answer: the reviewer, the planner, and the marionette's map. Those requests send the shape as `response_format: {"type": "json_schema", ...}`. Nearly every OpenAI-compatible server takes it. Older vLLM and llama.cpp builds, and some self-hosted gateways, reject it and accept only `{"type": "json_object"}`. Set `structuredOutputs: false` there: ```json { "providers": [ { "id": "homelab", "baseURL": "http://192.168.1.40:8000/v1", "structuredOutputs": false } ] } ``` The request then goes out as `json_object` with the shape written into the system prompt. The model reads the shape instead of being held to it, which is less reliable but works. Change this only if your server rejects a request and names `response_format`. ## Declare model limits Empryo cannot know what a self-hosted gateway actually serves, so an undeclared model is assumed to have a **128k context window**. If your model is smaller, auto-compaction aims at the wrong ceiling: the trigger waits for ~70% of 128k while the gateway rejects the request far earlier, it looks like compaction "doesn't work". Declare the real limits per model: ```json { "providers": [{ "id": "local", "baseURL": "http://localhost:8080/v1", "models": [ { "id": "qwen-32k", "name": "Qwen 32k", "contextWindow": 32768, "maxOutputTokens": 4096 } ] }] } ``` - `contextWindow` sizes the context gauge and fires auto-compaction at the right point. - `maxOutputTokens` sizes the output reserve (how much room a reply keeps). - If the provider's `/v1/models` endpoint reports a window, the live value wins. Your declaration is the offline/undeclared fallback. - Per-model override without touching the provider block: `"contextWindowOverrides": { "local/qwen-32k": 32768 }` at the config root. ## More examples #### Local server (no key) ```json { "providers": [{ "id": "local", "name": "Local LLM", "baseURL": "http://localhost:8080/v1", "models": ["llama-3-70b"] }] } ``` #### Corporate gateway with dynamic model list ```json { "providers": [{ "id": "corp", "name": "Corp Gateway", "baseURL": "https://llm.internal.corp.com/v1", "envVar": "CORP_LLM_KEY", "modelsAPI": "https://llm.internal.corp.com/v1/models" }] } ``` #### Multiple providers at once ```json { "providers": [ { "id": "together", "baseURL": "https://api.together.xyz/v1", "envVar": "TOGETHER_API_KEY", "models": ["meta-llama/Llama-3-70b"] }, { "id": "cerebras", "baseURL": "https://api.cerebras.ai/v1", "envVar": "CEREBRAS_API_KEY", "modelsAPI": "https://api.cerebras.ai/v1/models" } ] } ``` ## Thinking / reasoning For models that support reasoning over an OpenAI-compatible endpoint, add a `reasoning` block. Three styles are supported - use the one your provider accepts: ```json { "providers": [{ "id": "dashscope", "baseURL": "https://dashscope.aliyuncs.com/compatible-mode/v1", "envVar": "DASHSCOPE_API_KEY", "models": ["qwen-plus"], "reasoning": { "enabled": true, "budget": 8192 } }] } ``` | Style | Fields | When to use | |-------|--------|-------------| | OpenAI-style | `effort: "low" \| "medium" \| "high" \| "xhigh" \| "none"` | OpenAI-compatible reasoning models | | DashScope-style | `enabled: true`, `budget: 8192` | Alibaba Qwen, DashScope APIs | | Raw | `extraParams: { ... }` | Anything else - fields are forwarded verbatim | | Automatic | `auto: true` | You do not want to write any of the above - see below | The fields are injected into every request body. Raw `extraParams` override all other keys on collision. ### `auto` - let Empryo pick the body A `reasoning` block is a declaration: without one, Empryo asks your endpoint for nothing, and a reasoning model it resells thinks at whatever the endpoint defaults to - often invisibly, because nothing requested the thinking in the first place. If you would rather not work out which knob your backend speaks, turn on `auto`: ```json { "providers": [{ "id": "my-gateway", "baseURL": "https://llm.internal.corp.com/v1", "envVar": "MY_GATEWAY_KEY", "reasoning": { "auto": true } }] } ``` Empryo then looks each model up in the [models.dev](https://models.dev) catalog and sends the standard OpenAI-compatible knob **only** where the catalog says that model reasons - clamped to the effort ladder the catalog declares, plus the vendor's thinking toggle where one exists (`enable_thinking` for Qwen, `thinking: { type }` for GLM and Kimi). A model the catalog does not know, or knows as non-reasoning, is left completely alone. `auto` is **off by default and changes nothing for a provider that does not set it**: without it, requests are byte-for-byte what they always were. It also yields to any explicit declaration - set `effort`, `enabled`, `budget` or `extraParams` and those are used instead, because you have already answered the question `auto` exists to answer. `/effort` and `--effort` keep working either way, an effort you pick beats the automatic default. Use it when your provider resells a reasoning model and you never see any thinking. ### How `effort` is sent By default `effort` goes out as the flat OpenAI field, `{ "reasoning_effort": "high" }`, the field every OpenAI-compatible endpoint speaks. If your backend instead wants the nested OpenRouter shape, set `effortStyle`: ```json { "reasoning": { "effort": "high", "effortStyle": "nested" } } ``` | `effortStyle` | Sends | |---|---| | `"flat"` *(default)* | `{ "reasoning_effort": "high" }` | | `"nested"` | `{ "reasoning": { "effort": "high" } }` | | `"both"` | both keys at once | Only reach for `"both"` if a router genuinely needs it. Strictly-validating backends reject a request carrying both keys outright, *"Pass either `reasoning` (nested object) or `reasoning_effort` (flat field), not both."* ## Extra request-body parameters Some backends turn a capability on with a field in the request body rather than with a tool. DashScope reads `enable_search`, Zhipu reads `web_search`, and a gateway you run yourself reads whatever you taught it. `extraBody` sends the JSON you write with every chat request to that provider. ```json { "providers": [{ "id": "dashscope", "baseURL": "https://dashscope.aliyuncs.com/compatible-mode/v1", "envVar": "DASHSCOPE_API_KEY", "models": ["qwen-plus"], "extraBody": { "enable_search": true } }] } ``` The desktop app has the same field in its provider dialog, below Models. It checks the JSON as you type and keeps the save button off until the JSON parses. Empryo merges these keys last, after everything it built for the request, so they win against anything they collide with. A key your API rejects fails the whole request rather than being ignored, which is what a raw passthrough has to mean. `reasoning.extraParams` is the older field and it is not the same thing. Those params describe thinking, so they stop being sent when effort is `off`. `extraBody` goes out at every effort level. Put thinking schemas in one and everything else in the other. To send a param on a built-in lane instead of a custom provider, use the [`extraBody` map](/docs/reference/configuration#extra-request-body-parameters) in your config, keyed by provider or model id. ## Disable without removing Park a provider you are not using right now: click the power button next to it in the model picker's provider manager, or set `"disabled": true` on its config entry. A parked provider keeps its config, key binding and model list, but Empryo stops registering it - its models leave every picker and nothing calls its endpoints. The row stays in the manager, grayed out, one click from coming back. Useful when a local server (Ollama, LM Studio) is offline and you are tired of its models cluttering the picker, or when you switch between endpoints and want the spare one dormant instead of deleted. ## Scope - **Global** - `~/.empryo/config.json` applies everywhere. - **Project** - `.empryo/config.json` overrides global entries with the same `id`. ## Conflicts If your `id` matches a built-in (e.g. `"anthropic"`), Empryo renames it to `{id}-custom` - the built-in is never replaced. Custom providers always show `[custom]` in `/keys` and `--list-providers`.