Add a custom OpenAI-compatible endpoint

Connect any OpenAI- or Anthropic-compatible API, such as OpenRouter or a gateway you host, and use its models next to your other providers. Available on every plan. Your vault must be unlocked, because the key is encrypted in your browser before it's saved.

OpenLLM's servers call the endpoint for you, so it must be reachable from the public internet. An address on localhost or your home network won't work here. For servers on your own machine, use Local models instead.

Model kinds

When you enable a model, you choose its kind. The kind decides which requests can use it.

Kind	Label in the app	What OpenLLM sends
`chat_openai`	Chat · OpenAI wire	Chat requests in OpenAI format
`chat_anthropic`	Chat · Anthropic wire	Chat requests in Anthropic Messages format
`embeddings`	Embeddings (OpenAI wire)	Embedding requests
`image_openai`	Image · OpenAI /v1/images/generations	Image generation requests

The app suggests a kind for each model from its id. Check it before you save.

Add an endpoint

Open Providers and select the Custom tab.
Select Add custom API.
Fill in the form:
- Name: lowercase letters, digits, and hyphens, up to 63 characters, for example home-lab. It becomes part of every model id and must be unique.
- Base URL: choose OpenRouter, or Custom base URL… and enter the host root, such as https://api.example.com. Leave off /v1; OpenLLM adds it.
- Auth header for /v1/models: Bearer for OpenAI-style gateways, or x-api-key for Anthropic-style ones. This is only used for the discovery step.
- API key: required.
Select Discover models. OpenLLM calls <base URL>/v1/models and lists what the endpoint returns. Nothing is saved yet.
Tick the models you want. All models start unticked. Use the search box, Select all, or Clear for long lists, and set each model's kind.
Select Save N models.

There's no CLI command for adding an endpoint.

Use the models

Each enabled model gets an id in the form custom:<name>/<model>, where <model> is the id the endpoint reported. For example, custom:home-lab/qwen3-coder. Ids that contain slashes keep them, so custom:openrouter/anthropic/claude-opus-5 is valid.

Use these ids like any other model: in requests, in fallback chains on Configuration, and in model pickers across the app. They're listed in GET https://www.openllm.sh/v1/models.

Custom endpoint requests always run on the hosted gateway. A request sent to the daemon on your device is forwarded to the gateway for these models.

OpenLLM forwards Idempotency-Key and x-openllm-pin-model to custom endpoints. Built-in providers never receive these headers.

Update or remove an endpoint

To add models, remove models, or change a model's kind, open the endpoint's ⋯ menu and select Update models. Select Run discovery, adjust the ticks and kinds, then select Save N models.
To delete the endpoint, open the ⋯ menu, select Remove endpoint, and confirm. Requests that name its models fail until you add it again. In a fallback chain, those models are skipped.

How the key is stored

Your browser encrypts the base URL and API key together with your vault key and sends only the ciphertext. OpenLLM's servers decrypt them in memory for each request. During discovery, the key is sent to OpenLLM's servers for that one call so they can query the endpoint, and it isn't stored. The list of enabled models is stored as plain account settings. See Security and vault.

Check that it works

Call GET https://www.openllm.sh/v1/models and look for ids starting with custom:<name>/.

Send a request with one of them:

curl https://www.openllm.sh/v1/chat/completions \
  -H "Authorization: Bearer $OPENLLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "custom:home-lab/qwen3-coder", "messages": [{"role": "user", "content": "Hi"}]}'

Check that the x-openllm-resolved-model response header names your custom model.

Common problems

Symptom	Cause	Fix
`base_url must start with http:// or https://`	The base URL has no scheme.	Enter the full URL, starting with `https://` or `http://`.
`Failed to reach <url>`	OpenLLM's servers can't connect to the host. It may be private, offline, or blocked.	Use a public address, or use Local models for a server on your machine.
`Upstream returned 401` (or another status)	The endpoint rejected the discovery call.	Check the API key and the Auth header for /v1/models setting.
Endpoint returned no models	`/v1/models` returned an empty list.	Check the base URL. Some gateways only list models for keys with access to them.
A chat model fails with a format error	The model has the wrong kind.	Use Update models to switch it to Chat · OpenAI wire or Chat · Anthropic wire.

Providers

Routing and fallback

Local models

API reference

Last updated on

Add a custom endpoint