Clients

Any OpenAI or Anthropic client

Connect any OpenAI- or Anthropic-compatible tool or SDK to OpenLLM with the right base URL, your sk-llm key in either header, and a model id or alias.

OpenLLM serves OpenAI- and Anthropic-compatible endpoints, so any client that lets you change the base URL can use it. You need an sk-llm-… key from Devices & keys. All plans.

Connection details

SettingValue
OpenAI base URLhttps://www.openllm.sh/v1
Anthropic base URLhttps://www.openllm.sh (the SDK adds /v1/messages)
Key headerAuthorization: Bearer sk-llm-… or x-api-key: sk-llm-…
ModelAn alias (ultra, plus, lite) or a model id from GET /v1/models

The OpenAI-style endpoints include /v1/chat/completions, /v1/responses, /v1/embeddings, and /v1/models. The Anthropic-style endpoint is /v1/messages. For the full list, see API.

Connect your client

Store your key in an environment variable such as OPENLLM_API_KEY rather than in code.

curl https://www.openllm.sh/v1/chat/completions \
  -H "Authorization: Bearer $OPENLLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "plus",
    "messages": [{"role": "user", "content": "Reply with OK"}]
  }'

These are standard SDK calls with the base URL changed; nothing OpenLLM-specific is installed.

Choose a model

Set model to ultra, plus, or lite to use your fallback chains, or to a model id such as provider/model to use one model. Get ids from GET /v1/models. Change what the aliases resolve to in Configuration. See Routing and fallback.

Use the local daemon

If the OpenLLM daemon runs on the same machine, you can use it instead of the hosted gateway:

SettingValue
OpenAI base URLhttp://127.0.0.1:8787/v1
Anthropic base URLhttp://127.0.0.1:8787

The port is OPENLLM_DAEMON_PORT in ~/.openllm/.env if you changed it. The daemon listens only on 127.0.0.1 and does not check a key. If your SDK requires one, use your real sk-llm key, not a placeholder: when the daemon passes a whole request on to the hosted gateway (a chain with no subscription or local model), it forwards the Authorization header you sent. With no header, it uses the daemon's own key.

Use the daemon when your chains include subscription or local models. The hosted gateway cannot run those itself; it hands them to one of your online devices, and returns 403 subscription_requires_daemon or local_requires_daemon if none can serve them. See Devices and Local models.

Check that it works

Send the curl request above with -i. A 200 response carries the x-openllm-resolved-model header, which names the model that served it. The request also appears on Overview in the app.

Common problems

SymptomCauseFix
401Missing or invalid key, or a key that is not sk-llm-…Send your key in Authorization or x-api-key
Anthropic calls return 404/v1 was added to the Anthropic base URLUse https://www.openllm.sh without /v1
OpenAI calls return 404 or an HTML page/v1 is missing from the OpenAI base URLUse https://www.openllm.sh/v1
403 subscription_requires_daemonThe chain has only subscription models and no device can serve themStart the daemon, or add an API-key model to the chain

For error types and fixes, see Errors.

Last updated on

On this page