Installation
LLLM needs Python 3.10 or later. The distribution is called lllm-core; you
import it as lllm.
pip install "lllm-core[native]"
or with uv:
uv add "lllm-core[native]"
The base package has a single dependency (Pydantic) and gives you the typed
tactic boundary. The native extra adds what agents need to call models.
Extras
| Extra | Adds | Install it for |
|---|---|---|
native |
LiteLLM, YAML configs | agents calling models (most users) |
server |
FastAPI, Uvicorn | serving tactics with lllm serve |
client |
httpx | calling served tactics with RemoteTactic |
sandbox |
Jupyter client and kernel | the code sandbox |
pydantic-ai |
Pydantic AI | running Pydantic AI agents as tactics |
secure |
keyring | storing API keys with lllm setup |
tools |
OpenAI SDK, Playwright | the experimental browser-use agent |
network |
SSSN | channels between systems; adds lllm network |
shell |
AAAX | strategies that serve many tactics; adds lllm shell |
all |
everything above |
pip install "lllm-core[native,server]"
Connect a model
Agents call models through LiteLLM, so any provider it supports works. Set the provider's key in the environment and name the model:
export OPENAI_API_KEY=sk-... # or ANTHROPIC_API_KEY, GEMINI_API_KEY, ...
from lllm import Tactic
print(Tactic.quick("Say hi", model="anthropic/claude-sonnet-4-5").content)
Model names follow LiteLLM: gpt-4o-mini, anthropic/claude-sonnet-4-5,
gemini/gemini-2.5-flash, ollama/llama3.2, and so on. In a tactic, set the
model once in the config ("global": {"model_name": ...}) or per agent.
Local models
Point an agent at any OpenAI-compatible server (vLLM, SGLang, Ollama,
llama.cpp) with the openai/ prefix and the server's address:
from lllm import Tactic
reply = Tactic.quick(
"Say hi",
model="openai/Qwen/Qwen2.5-7B-Instruct",
api_base="http://localhost:8000/v1",
api_key="local",
)
print(reply.content)
Extra keyword arguments to Tactic.quick() are sent with every model call;
in a config they go under model_args. Models covers providers,
settings, and custom invokers.
Check the install
You can run an agent without any key: lllm.testing answers
model calls locally.
from lllm import Tactic
from lllm.testing import ScriptedModel
with ScriptedModel(["LLLM is installed."]):
print(Tactic.quick("Are you there?").content)
#> LLLM is installed.
The command line
Installing LLLM adds the lllm command:
lllm --help
lllm create my-app # start a project from a template
lllm serve my-app # serve its tactic over HTTP
lllm hub validate my-app
See the CLI reference.
From source
git clone https://github.com/Productive-Superintelligence/lllm
cd lllm
pip install -e ".[all,dev,docs]"
pytest