Any OpenAI or Anthropic client
Connect any OpenAI- or Anthropic-compatible tool or SDK to OpenLLM with the right base URL, your sk-llm key in either header, and a model id or alias.
OpenLLM serves OpenAI- and Anthropic-compatible endpoints, so any client that lets you change the base URL can use it. You need an sk-llm-… key from Devices & keys. All plans.
Connection details
| Setting | Value |
|---|---|
| OpenAI base URL | https://www.openllm.sh/v1 |
| Anthropic base URL | https://www.openllm.sh (the SDK adds /v1/messages) |
| Key header | Authorization: Bearer sk-llm-… or x-api-key: sk-llm-… |
| Model | An alias (ultra, plus, lite) or a model id from GET /v1/models |
The OpenAI-style endpoints include /v1/chat/completions, /v1/responses, /v1/embeddings, and /v1/models. The Anthropic-style endpoint is /v1/messages. For the full list, see API.
Connect your client
Store your key in an environment variable such as OPENLLM_API_KEY rather than in code.
curl https://www.openllm.sh/v1/chat/completions \
-H "Authorization: Bearer $OPENLLM_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "plus",
"messages": [{"role": "user", "content": "Reply with OK"}]
}'import os
from openai import OpenAI
client = OpenAI(
base_url="https://www.openllm.sh/v1",
api_key=os.environ["OPENLLM_API_KEY"],
)
response = client.chat.completions.create(
model="plus",
messages=[{"role": "user", "content": "Reply with OK"}],
)
print(response.choices[0].message.content)import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://www.openllm.sh/v1",
apiKey: process.env.OPENLLM_API_KEY,
});
const response = await client.chat.completions.create({
model: "plus",
messages: [{ role: "user", content: "Reply with OK" }],
});
console.log(response.choices[0].message.content);import os
from anthropic import Anthropic
client = Anthropic(
base_url="https://www.openllm.sh",
api_key=os.environ["OPENLLM_API_KEY"],
)
message = client.messages.create(
model="plus",
max_tokens=256,
messages=[{"role": "user", "content": "Reply with OK"}],
)
print(message.content[0].text)import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic({
baseURL: "https://www.openllm.sh",
apiKey: process.env.OPENLLM_API_KEY,
});
const message = await client.messages.create({
model: "plus",
max_tokens: 256,
messages: [{ role: "user", content: "Reply with OK" }],
});
console.log(message.content[0].type === "text" ? message.content[0].text : "");Many tools read the standard SDK variables. Set the pair for the API your tool speaks.
# OpenAI-compatible tools
export OPENAI_BASE_URL="https://www.openllm.sh/v1"
export OPENAI_API_KEY="sk-llm-…"
# Anthropic-compatible tools
export ANTHROPIC_BASE_URL="https://www.openllm.sh"
export ANTHROPIC_API_KEY="sk-llm-…"These are standard SDK calls with the base URL changed; nothing OpenLLM-specific is installed.
Choose a model
Set model to ultra, plus, or lite to use your fallback chains, or to a model id such as provider/model to use one model. Get ids from GET /v1/models. Change what the aliases resolve to in Configuration. See Routing and fallback.
Use the local daemon
If the OpenLLM daemon runs on the same machine, you can use it instead of the hosted gateway:
| Setting | Value |
|---|---|
| OpenAI base URL | http://127.0.0.1:8787/v1 |
| Anthropic base URL | http://127.0.0.1:8787 |
The port is OPENLLM_DAEMON_PORT in ~/.openllm/.env if you changed it. The daemon listens only on 127.0.0.1 and does not check a key. If your SDK requires one, use your real sk-llm key, not a placeholder: when the daemon passes a whole request on to the hosted gateway (a chain with no subscription or local model), it forwards the Authorization header you sent. With no header, it uses the daemon's own key.
Use the daemon when your chains include subscription or local models. The hosted gateway cannot run those itself; it hands them to one of your online devices, and returns 403 subscription_requires_daemon or local_requires_daemon if none can serve them. See Devices and Local models.
Check that it works
Send the curl request above with -i. A 200 response carries the x-openllm-resolved-model header, which names the model that served it. The request also appears on Overview in the app.
Common problems
| Symptom | Cause | Fix |
|---|---|---|
| 401 | Missing or invalid key, or a key that is not sk-llm-… | Send your key in Authorization or x-api-key |
| Anthropic calls return 404 | /v1 was added to the Anthropic base URL | Use https://www.openllm.sh without /v1 |
| OpenAI calls return 404 or an HTML page | /v1 is missing from the OpenAI base URL | Use https://www.openllm.sh/v1 |
403 subscription_requires_daemon | The chain has only subscription models and no device can serve them | Start the daemon, or add an API-key model to the chain |
For error types and fixes, see Errors.
Related
Last updated on
DeepSeek Harness
Route DeepSeek Harness (dsh) through the OpenLLM daemon on your machine by adding the OpenLLM bundle to a dsh profile with the dsh plugin command.
MCP server
Add the openllm MCP server to any MCP client over stdio to get gateway API tools, shared memory, subscription logins, and your marketplace integrations.