Make agents cheaper and faster

Most of what an agent spends money and time on is reading. It greps around a repo, opens files it does not need, and re-learns the same facts every session. OpenLLM ships two plugins that cut both: semantic code search so an agent finds the right file instead of scanning many, and persistent memory so it never re-derives what you already told it. Fewer tokens in the context means cheaper and faster requests, on any model.

Install the plugin

Both come from the OpenLLM CLI, which serves one MCP server exposing the gateway API, code and docs search, and memory:

curl -fsSL "https://openllm.sh/api/setup/cli/install.sh" | bash

After a sandboxed install, finish shell completion with ~/.openllm/bin/openllmc setup. In Claude Code, the same plugin also installs background hooks that index your repo on session start and refresh it after edits, so search stays current without you running anything. See Connect your tools.

Code search: read the right file, not every file

Instead of keyword grep, code search finds code by meaning. You index a repo once, then ask conceptual questions:

• index_codebase: build an embedding index of the repository.
• search_code: semantic search over it ("where is auth handled?").
• get_indexing_status: check progress.
• index_docs / search_docs: do the same for a documentation URL.
• clear_index: drop an index.

The payoff is token cost. A semantic hit lands the agent on the two files that matter, rather than feeding twenty files of grep output into the model's context to find them. Smaller context is a smaller bill and a faster response, and the effect compounds over a long agent run.

Memory: stop re-explaining yourself

Persistent memory lets an agent save a durable fact and recall it in a later session, so your preferences, stack decisions, and project constraints do not have to be retyped into every new thread.

• memory: save or forget a fact, optionally scoped to a project.
• recall: pull the relevant saved facts for the task at hand.

In Claude Code this is wired to hooks too: one pulls in similar memories when you submit a prompt, another extracts durable conclusions when a session ends. The result is an agent that starts each session already knowing the things you would otherwise spend tokens re-establishing.

Saved memories are encrypted at rest under your vault key and shared across keys derived from the same recovery phrase. See Security for how the vault works.

Web search on every model

The third thing that keeps an agent from guessing is a live web result. Claude Code's WebSearch works through OpenLLM on every subscription provider, not just Claude: when your client declares the search tool, the request is handed to that provider's own native search (Anthropic runs it directly; Codex uses hosted search; Grok searches the web and X server-side; Kimi uses Moonshot's builtin). The provider owns the search, so there is no extra round trip through the gateway.

For anything without native search, the openllmc CLI exposes a portable /v1/search as an MCP tool, which runs against the search provider you configure in the dashboard. Either way your agent gets grounded, cited answers on whichever model you happen to be using.