Configuration
A tactic's agents are described by a config: the model each agent uses, its system prompt, its settings and tools, and how many retries it gets. The config is data, a dict or a YAML file, so you can review, diff, and swap the shape of a system without touching the program that runs it.
from lllm import Tactic
class ReleaseNotes(Tactic):
name = "release_notes"
agent_group = ["writer", "editor"]
def call(self, changes: str) -> str:
writer, editor = self.agents["writer"], self.agents["editor"]
writer.open("draft")
writer.receive(f"Write release notes for: {changes}")
draft = writer.respond().content
editor.open("edit")
editor.receive(f"Tighten these release notes:\n{draft}")
return editor.respond().content
config = {
"global": { # (1)!
"model_name": "gpt-4o-mini",
"model_args": {"temperature": 0.2},
},
"agent_configs": [ # (2)!
{"name": "writer", "system_prompt": "You write release notes."},
{
"name": "editor",
"system_prompt": "You edit for brevity.",
"model_name": "gpt-4o", # (3)!
"model_args": {"max_tokens": 200},
},
],
}
notes = ReleaseNotes(config)
for name, agent in notes.agents.items():
print(name, agent.model, agent.model_args)
#> writer gpt-4o-mini {'temperature': 0.2}
#> editor gpt-4o {'temperature': 0.2, 'max_tokens': 200}
print(notes("faster startup; fixed the login timeout"))
#> Faster startup, and login no longer times out.
globalholds the defaults for every agent.- One entry per agent in the tactic's
agent_group, matched byname. An entry overridesglobal, andmodel_argsdicts are merged. - One agent can use another model without touching the rest.
The config is read once, when the tactic is built, and every call gets
fresh agents made from it. notes.agents holds the agents built at
construction, which is a quick way to check what a config produces.
Agent settings
| Key | Default | Meaning |
|---|---|---|
name |
required | The agent in agent_group this entry describes |
model_name |
required | A LiteLLM model name, such as gpt-4o-mini |
system_prompt |
The system prompt as text | |
system_prompt_path |
A registered prompt, such as analyst/system |
|
model_args |
{} |
Sent with every call: temperature, max_tokens, api_base, ... |
api_type |
completion |
response selects OpenAI's Responses API |
max_interrupt_steps |
5 | Tool rounds per respond() |
max_exception_retry |
3 | Repairs of replies that fail to parse |
max_llm_recall |
0 | Retries after a provider error |
max_rate_limit_retries |
8 | Retries after a rate limit |
tools |
[] |
Registered tools, tactics, and proxies (Tools) |
proxy |
API proxies and the code interpreter (Proxies) | |
skills |
Agent skills: local names, SKILL.md URLs, or "*" (Agents) |
|
context_manager |
Keeps the dialog inside the context window (Dialogs) | |
inference_hints |
Session and priority hints (Inference hints) |
Every agent needs model_name and one of system_prompt or
system_prompt_path, from its own entry or from global. Both kinds of
system prompt can have {placeholders}, filled when the agent opens a
dialog with open(alias, prompt_args={...}). The limits are explained in
the agent loop.
A config is checked when the tactic is built, before any model call. An
agent without a model or a system prompt, or an agent in agent_group
without an entry, raises ValueError naming it, and a system_prompt_path
that is not registered raises KeyError. A key LLLM does not recognize is
added to model_args, with a warning that names it, so a typo such as
temprature shows up in the logs instead of vanishing.
Global defaults
Each entry is merged onto global. Dicts merge key by key, while lists
and plain values replace. So one agent can add a model argument without
repeating the others, and an empty list removes a global one:
global:
model_name: gpt-4o-mini
model_args: {temperature: 0.2}
tools: [search, fetch_page]
agent_configs:
- name: researcher # both tools
system_prompt: You find sources.
- name: writer
system_prompt: You write the report.
model_args: {max_tokens: 800} # and temperature 0.2
tools: [] # no tools
Tactic-level keys
| Key | Read by | Meaning |
|---|---|---|
global |
the tactic | Defaults for every agent |
agent_configs |
the tactic | One entry per agent |
base |
resolve_config |
A config to inherit from (Inheritance) |
tactic_type |
build_tactic |
The registered tactic to build (Building by name) |
invoker |
the tactic | The invoker that calls models (default litellm) |
invoker_config |
the tactic | Settings passed to the invoker (default: the whole config) |
max_workers |
bcall, ccall |
How many calls run at once (default 4) |
Anything else at the top level is yours. The tactic keeps the whole config
as self.config, so call() can read a rubric, a word limit, or a list of
sources from it.
Configs in a project
In a project, configs are YAML files and prompts are Python files, in
folders that the project's lllm.toml lists (Packages
covers the manifest). lllm create makes this layout:
markets/
├── lllm.toml
├── configs/default.yaml
├── prompts/analyst.py
└── tactics/report.py
This script writes the first three files into the current folder, with the tactic in the script itself, and runs it:
from pathlib import Path
from lllm import Tactic, load_runtime, resolve_config
Path("lllm.toml").write_text('[package]\nname = "markets"\n')
Path("prompts").mkdir()
Path("prompts/analyst.py").write_text("""
from lllm import Prompt
system = Prompt(path="system", prompt="You analyze {market} markets.")
""")
Path("configs").mkdir()
Path("configs/default.yaml").write_text("""
global:
model_name: gpt-4o-mini
agent_configs:
- name: analyst
system_prompt_path: analyst/system
""")
class Report(Tactic):
name = "report"
agent_group = ["analyst"]
def call(self, question: str) -> str:
analyst = self.agents["analyst"]
analyst.open("report", prompt_args={"market": "energy"})
analyst.receive(question)
return analyst.respond().content
runtime = load_runtime() # (1)!
print(runtime.keys("config"))
#> ['markets.configs:default']
print(runtime.keys("prompt"))
#> ['markets.prompts:analyst/system']
report = Report(resolve_config("default")) # (2)!
print(report("Why did oil prices fall this week?"))
#> Oil fell on rising inventories and weaker demand forecasts.
load_runtime()findslllm.tomland registers what its folders hold, each under the package name.resolve_config()returns the config as a dict. The tactic then loadsanalyst/systemfrom the runtime.
Names follow the files:
- A config is named by its path under
configs/without.yaml:default,experiments/fast. - A prompt is named by its file and its
path:Prompt(path="system")inprompts/analyst.pyisanalyst/system. - A bare name looks in your own package. Use
pkg:namefor a dependency's config or prompt, such asshared:defaultorshared:analyst/system.
load_runtime() reads the file named by the LLLM_CONFIG environment
variable if it is set, and otherwise the first lllm.toml in the working
directory or its parents; load_runtime("path/to/lllm.toml") picks one
explicitly. It also loads the packages installed in the project's
lllm_packages/ folder and in ~/.lllm/packages/. Nothing is read when you
import lllm, and a YAML file is read the first time it is used.
Building a tactic by name
build_tactic(config) constructs the registered tactic named by the
config's tactic_type, so the config chooses the program as well as the
agents:
from lllm import Tactic, build_tactic, register_tactic_class
@register_tactic_class # (1)!
class Translate(Tactic):
name = "translate"
agent_group = ["translator"]
def call(self, text: str) -> str:
translator = self.agents["translator"]
translator.open("t")
translator.receive(text)
return translator.respond().content
config = {
"tactic_type": "translate",
"global": {"model_name": "gpt-4o-mini"},
"agent_configs": [
{"name": "translator", "system_prompt": "Translate into French."}
],
}
tactic = build_tactic(config)
print(type(tactic).__name__, tactic("Good morning"))
#> Translate Bonjour
- In a project,
load_runtime()registers everyTacticsubclass with anameit finds undertactics/. The decorator does the same by hand.
So a project starts with load_runtime() and then
build_tactic(resolve_config("default")), which is what lllm create puts
in main.py. Pass name="..." to build another tactic with the same
config. Other keyword arguments, such as observer=, go to the tactic's
constructor.
Inheritance
A config can name a base config and change only what differs. The merge
is the same as for global: dicts merge and everything else replaces.
from pathlib import Path
from lllm import load_runtime, resolve_config
Path("lllm.toml").write_text('[package]\nname = "markets"\n')
Path("configs/experiments").mkdir(parents=True)
Path("configs/default.yaml").write_text("""
global:
model_name: gpt-4o
model_args: {temperature: 0.1}
agent_configs:
- name: analyst
system_prompt: You analyze markets.
""")
Path("configs/experiments/fast.yaml").write_text("""
base: default
global:
model_name: gpt-4o-mini
model_args: {max_tokens: 500}
""")
load_runtime()
config = resolve_config("experiments/fast")
print(config["global"]["model_name"])
#> gpt-4o-mini
print(config["global"]["model_args"]) # (1)!
#> {'temperature': 0.1, 'max_tokens': 500}
print([agent["name"] for agent in config["agent_configs"]]) # (2)!
#> ['analyst']
model_argsis a dict, so the two files' arguments are merged.fast.yamlhas noagent_configs, so it keeps its base's list.
A base can have its own base, so experiments can stack. The result has no
base key, and a cycle raises ValueError. load_config(name) returns a
file as written, without following base.
A config meant to be shared from a package should name its parent with the
package prefix (base: shared:default), so it resolves from any project
that depends on it. To pin a dependency's config with your own changes, use
vendor_config, which resolves the chain and merges your overrides on top:
from lllm.runtimes.native.core.config import vendor_config
config = vendor_config(
"shared:default", {"global": {"model_name": "gpt-4o"}}
)
API keys and environment variables
Configs name models; they do not hold secrets. Keys and paths come from the environment:
| Variable | Read by | Purpose |
|---|---|---|
OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY, ... |
LiteLLM | Provider keys (Models) |
LLLM_CONFIG |
load_runtime() |
An lllm.toml file, or a folder with one, to load instead of searching |
LLLM_CONFIG lets one project run with another manifest, for example
LLLM_CONFIG=deploy/lllm.prod.toml python main.py. A package can also
declare the keys it needs: lllm setup asks for the missing ones and
stores them in the OS keyring (with the secure extra) or in .env.local,
and lllm serve reads them from there. See Packages.