Core concepts

A few ideas show up everywhere in OpenLLM. Once these click, the rest of the docs read quickly.

Your OpenLLM key

One key, shaped like sk-llm-…, that every tool uses. It authenticates you to the gateway and, on each request, is what derives the in-memory unlock for your stored credentials. Treat it like a password. You can create and revoke keys under Devices & keys in the dashboard.

Providers

A provider is one AI account you connect: an API key like Anthropic or OpenAI, or a subscription account like ChatGPT or Claude Code over OAuth. You connect providers once under Providers, and OpenLLM uses them on your behalf. See Providers for the full list and the two connection types.

Tiers, which are fallback chains

A tier and a fallback chain are the same thing seen from two sides: the tier is the alias you use as the model name, and the chain is the ordered list of providers behind that name. Claude first, then GPT, then Grok, for instance. If the first is down or rate-limited, OpenLLM tries the next. This is what keeps a long agent task from dying on a single 429.

OpenLLM ships three presets:

• lite for fast, cheap work
• plus for everyday coding and chat
• ultra for the hardest tasks

Those are just defaults. You can rename them, change what backs them, or create your own aliases with whatever names fit how you work (review, scaffold, cheap-batch, anything).

You call a tier, and OpenLLM resolves it to a concrete provider and model through its chain. The point is indirection: you can change which model backs ultra without touching a single line of your code or any tool config. See Fallback chains for how resolution works.

You can still call a specific model by its full id (for example anthropic/claude-...) when you want to pin one exactly. In a coding tool that is one slash command away: /model plus the tier or model name switches the thread on the spot. See Try and switch models.

The dashboard

One place that shows every request across every provider, with cost and failure rates. Because OpenLLM adds no markup, the cost you see is the cost your provider charged, which is the whole point of having it in one view.

For subscription usage it goes a step further: next to what you actually paid, it shows what the same tokens would have cost at the provider's metered API price, so your savings are a number you can see, not a feeling. See Track usage and savings.