API reference

OpenLLM speaks the OpenAI and Anthropic wire formats, so most SDKs work after you change the base URL and the key.

Base URL: https://www.openllm.sh/v1

To run the examples on this page, export your key first:

export OPENLLM_API_KEY=sk-llm-YOUR_KEY

Authentication

Send your OpenLLM key in either header:

Authorization: Bearer sk-llm-YOUR_KEY
x-api-key: sk-llm-YOUR_KEY

• Keys look like sk-llm-{id}.{secret}. Only values that start with sk-llm- are read as keys.
• Create keys on the Devices & keys page. Keys have no scopes and no expiry. See Security for what a key can unlock.
• A dashboard session (being signed in to the web app) is not enough for /v1. Only GET /v1/models and GET /v1/models/{id} accept a session; every other endpoint answers 401 auth_error without an API key. (/v1/realtime is the exception: the hosted gateway always refuses it.)
• A key minted before a vault reset stops working. See This key belongs to a previous vault.

Authentication failures use the auth_error type. See Errors.

Endpoints

No /v1 endpoint requires the Pro plan. On the Free plan, every metered endpoint stops with 402 free_tier_exhausted once your monthly tokens are used up.

Device only means the request runs on the daemon on one of your machines. The hosted gateway hands /v1/responses/compact to an online device and answers 501 subscription_requires_daemon when none can take it. It never serves /v1/realtime: that call always gets 403 subscription_requires_daemon, so connect to the daemon directly. A request whose chain resolves only to subscription or local models is handed to a device the same way; with none able to serve it, you get 403 subscription_requires_daemon or 403 local_requires_daemon. See Routing.

Chat completions

curl https://www.openllm.sh/v1/chat/completions \
  -H "Authorization: Bearer $OPENLLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "plus",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Messages

The Anthropic Messages format. POST /v1/messages/count_tokens takes the same body and returns { "input_tokens": … }.

curl https://www.openllm.sh/v1/messages \
  -H "x-api-key: $OPENLLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "plus",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Responses

The OpenAI Responses format. input accepts a string or an array of typed items. POST /v1/responses/compact is Codex's own compaction call; a device signed in to ChatGPT serves it, and the hosted gateway answers 501 subscription_requires_daemon when none can.

curl https://www.openllm.sh/v1/responses \
  -H "Authorization: Bearer $OPENLLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "plus", "input": "Hello!"}'

Embeddings

Set dimensions to a width the model supports.

curl https://www.openllm.sh/v1/embeddings \
  -H "Authorization: Bearer $OPENLLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "openai/text-embedding-3-small", "input": "The quick brown fox"}'

Images

Generate an image:

curl https://www.openllm.sh/v1/images/generations \
  -H "Authorization: Bearer $OPENLLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "YOUR_MODEL", "prompt": "A cute cat", "n": 1, "size": "1024x1024"}'

Edit an image with POST /v1/images/edits. It accepts exactly one reference image as a multipart file or a JSON data: URL. A remote image URL, a mask, or a second image is rejected. Editing works only through the chatgpt and grok subscription providers, so a device signed in to one of them serves the request. Any other provider returns 400 image_edit_unsupported_provider.

curl https://www.openllm.sh/v1/images/edits \
  -H "Authorization: Bearer $OPENLLM_API_KEY" \
  -F "[email protected]" \
  -F "prompt=Add a red hat" \
  -F "model=chatgpt/gpt-image-2"

Videos

Video generation is an async job: create it, poll until "status": "completed", download the clip, and optionally delete the job.

# 1. Create the job
curl https://www.openllm.sh/v1/videos \
  -H "Authorization: Bearer $OPENLLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "YOUR_MODEL", "prompt": "A cat playing with yarn", "seconds": "4", "size": "1280x720"}'

# 2. Poll the job
curl https://www.openllm.sh/v1/videos/VIDEO_ID \
  -H "Authorization: Bearer $OPENLLM_API_KEY"

# 3. Download the MP4
curl https://www.openllm.sh/v1/videos/VIDEO_ID/content \
  -H "Authorization: Bearer $OPENLLM_API_KEY" \
  --output video.mp4

# 4. Cancel or delete a job you no longer need
curl -X DELETE https://www.openllm.sh/v1/videos/VIDEO_ID \
  -H "Authorization: Bearer $OPENLLM_API_KEY"

• Downloading before the job finishes returns 409 video_not_ready.
• A seconds value the model does not support is refused with the accepted values named.
• A job running on a subscription provider finishes on the device that runs it. You can't cancel it through the gateway (501 subscription_requires_daemon).

Audio

Transcribe audio (multipart, or JSON with file as a base64 data: URL):

curl https://www.openllm.sh/v1/audio/transcriptions \
  -H "Authorization: Bearer $OPENLLM_API_KEY" \
  -F "[email protected]" \
  -F "model=YOUR_MODEL"

Generate speech. The response body is the audio file:

curl https://www.openllm.sh/v1/audio/speech \
  -H "Authorization: Bearer $OPENLLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "YOUR_MODEL", "input": "Hello, world!", "voice": "alloy"}' \
  --output speech.mp3

Search

Runs a web search with the search provider you connected on Providers. The body takes query (required), max_results, and country (ISO 3166 code). The response is { "query", "results": [{ "title", "url", "snippet", "page_age" }] }.

curl https://www.openllm.sh/v1/search \
  -H "Authorization: Bearer $OPENLLM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "OpenLLM gateway", "max_results": 5}'

Without a search provider key you get 404 search_credential_missing.

Models

GET /v1/models lists the models and aliases your account can use right now, in the OpenAI list format. GET /v1/models/{id} returns one entry or 404 not_found_error. This is the live source for model ids; other pages link here instead of listing them.

curl https://www.openllm.sh/v1/models \
  -H "Authorization: Bearer $OPENLLM_API_KEY"

Model ids and aliases

• A model id is provider/model, for example openai/text-embedding-3-small. Custom endpoints use custom:<name>/<model>.
• ultra, plus, and lite are aliases for fallback chains. If a model fails before it sends any output, OpenLLM tries the next model in the chain.
• To lock a request to one model and skip the chain, send x-openllm-pin-model: provider/model.

See Routing for how chains are built and edited.

Streaming

Set "stream": true on /v1/chat/completions, /v1/messages, or /v1/responses. The response is server-sent events (text/event-stream) in the format of the endpoint you called.

• Keepalive: an idle stream gets a keepalive every 15 seconds. /v1/messages sends an Anthropic ping event; the other endpoints send an SSE comment line.
• Fallback: OpenLLM can switch models only before the first output. Once a stream has started, an error ends the stream; there is no fallback.
• Time limits: chat, messages, responses, embeddings, images, and video downloads can run up to 800 seconds. Audio requests and video create, poll, and delete calls can run up to 300 seconds.

Idempotency

To make retries safe, send an Idempotency-Key header on /v1/chat/completions, /v1/messages, or /v1/responses.

• A retry replays the first response only when the key and the request body are byte-identical. A new body with the same key is a new request.
• Only successful responses are stored, for 10 minutes. Errors are never replayed.
• Responses larger than 1.5 MB are not stored. A non-streamed first response that is too large carries x-openllm-cache: skip-too-large; a streamed one reads miss.
• A replay carries idempotent-replayed: true and x-openllm-cache: hit.
• Every replay is one JSON body, marked x-openllm-replay-shape: json, even when the first request streamed.
• Responses served by one of your devices are not stored. /v1/responses/compact ignores the header.

Headers

Request headers:

Response headers:

Errors

Every error uses one envelope. The machine-readable code is type:

{ "error": { "message": "…", "type": "free_tier_exhausted" } }

Some errors add fields such as param, issues, or tokens_used. Errors that come from a provider keep that provider's own type. See Errors for every type and what to do about it.

OpenAPI spec

• JSON: https://www.openllm.sh/api/swagger. Paths in the spec are relative to the /api server.
• Interactive console: https://www.openllm.sh/api/swagger/ui
• Offline: openllm api --spec prints the same document, embedded in the CLI.

Local daemon endpoint

The daemon serves the same API on your machine at http://127.0.0.1:8787/v1. The port is OPENLLM_DAEMON_PORT; the installer picks a nearby free port when 8787 is taken, and openllm config --json shows the one in use.

• It listens on 127.0.0.1 only and needs no API key from the caller. It uses the key it was paired with. If you send your own Authorization: Bearer key, API-key requests it forwards to the cloud are recorded against that key.
• It serves chat completions, messages, token counts, responses (including /responses/compact), embeddings, images, audio, and videos. Subscription and local models run on the machine; API-key models are forwarded to the hosted gateway. GET /v1/models is passed through to the hosted gateway.
• Your chains still come from your account: the daemon asks the hosted gateway for a signed routing plan for each request.
• Realtime voice is a WebSocket at ws://127.0.0.1:<port>/v1/realtime?provider=…&model=…&voice=…. It requires Authorization: Bearer with the key the daemon is paired with, so a browser can't open it directly.

Last updated on