Core concepts

A few ideas show up everywhere in OpenLLM. Once these click, the rest of the docs read quickly.

Your OpenLLM key

One key, shaped like sk-llm-…, that every tool uses. It authenticates you to the gateway and, on each request, is what derives the in-memory unlock for your stored credentials. Treat it like a password. You can create and revoke keys under Keys in the dashboard.

Providers

A provider is one AI account you connect: an API key like Anthropic or OpenAI, or a subscription account like ChatGPT or Claude Code over OAuth. You connect providers once under Providers, and OpenLLM uses them on your behalf. See Providers for the full list and the two connection types.

Tiers

Tiers are simple aliases you use as the model name:

• lite for fast, cheap work
• plus for everyday coding and chat
• ultra for the hardest tasks

You call a tier, and OpenLLM resolves it to a concrete provider and model through your fallback chain. The point is indirection: you can change which model backs ultra without touching a single line of your code or any tool config.

You can still call a specific model by its full id (for example anthropic/claude-...) when you want to pin one exactly.

Fallback chains

A fallback chain is the ordered list of providers behind a tier. Claude first, then GPT, then Gemini, for instance. If the first is down or rate-limited, OpenLLM tries the next. This is what keeps a long agent task from dying on a single 429. See Fallback chains.

The dashboard

One place that shows every request across every provider, with cost and failure rates. Because OpenLLM adds no markup, the cost you see is the cost your provider charged, which is the whole point of having it in one view.