Agents
An agent is a model plus the loop that calls it. It keeps its own dialogs, runs the tools the model asks for, repairs replies that fail to parse, and records what every call cost. Your program decides what to ask; the agent makes each ask reliable.
| Part | What it is |
|---|---|
model, model_args |
Any LiteLLM model name, and settings such as temperature |
system_prompt |
A prompt that starts every dialog the agent opens |
| Dialogs | The agent's conversations, each under an alias (Dialogs) |
| Tools | Functions the model can call, from prompts or the agent's config (Tools) |
| Limits | How many tool rounds, repairs, and provider retries one call may use |
Here is an agent holding a conversation. It remembers the first turn because both turns go into the same dialog:
from lllm import Tactic
agent = Tactic.quick(
system_prompt="You are a helpful assistant.", model="gpt-4o-mini"
)
agent.open("chat")
agent.receive("My name is Ada.")
print(agent.respond().content)
#> Nice to meet you, Ada!
agent.receive("What is my name?")
print(agent.respond().content)
#> Your name is Ada.
Creating agents
There are three ways to get an agent, from quickest to most structured.
Tactic.quick() builds a one-off agent. With a question, it returns the
reply; without one, it returns the agent:
from lllm import Tactic
reply = Tactic.quick("What is the capital of France?", model="gpt-4o-mini")
print(reply.content)
#> The capital of France is Paris.
agent = Tactic.quick(
system_prompt="Answer in one word.", model="gpt-4o-mini"
)
print(agent.name, agent.model)
#> assistant gpt-4o-mini
In a tactic, agents are declared by name and described by config. Every call of the tactic gets fresh agents, so calls never share state. This is the usual way to use agents; see Tactics:
class Review(Tactic):
agent_group = ["reviewer"]
def call(self, code: str) -> str:
reviewer = self.agents["reviewer"]
...
review = Review({
"global": {"model_name": "gpt-4o-mini"},
"agent_configs": [
{"name": "reviewer", "system_prompt": "You review code."}
],
})
Directly, Agent is a dataclass you can build yourself, for example in a
script or a notebook:
from lllm import Agent, Prompt
from lllm.runtimes.native.invokers import build_invoker
agent = Agent(
name="writer",
system_prompt=Prompt(
path="writer/system", prompt="You write clear notes."
),
model="gpt-4o-mini",
llm_invoker=build_invoker({"invoker": "litellm"}),
model_args={"temperature": 0.2},
)
agent.open("draft")
agent.receive("Write a one-line project update.")
print(agent.respond().content)
#> The docs now run every example and check its output.
Conversations
An agent can keep several dialogs at once, each under an alias, and works on the active one. Use one alias per line of work, so histories never mix:
from lllm import Tactic
agent = Tactic.quick(
system_prompt="You are a careful planner.", model="gpt-4o-mini"
)
agent.open("plan") # (1)!
agent.receive("Plan a launch checklist in three steps.")
plan = agent.respond().content
agent.open("risks") # (2)!
agent.receive(f"List the biggest risk in this plan:\n{plan}")
print(agent.respond().content)
#> The biggest risk is skipping the rollback test.
agent.switch("plan") # (3)!
print(agent.active_alias, list(agent.dialogs))
#> plan ['plan', 'risks']
open()starts a dialog with the agent's system prompt and makes it active.- Opening a second dialog switches to it; the first keeps its history.
switch()goes back.close(alias)removes a dialog, andfork(alias, child_alias)branches one (Dialogs).
| Method | Does |
|---|---|
open(alias, prompt_args=None) |
Start a dialog; prompt_args fill the system prompt's variables |
receive(text) |
Add a user message |
receive_prompt(prompt, args) |
Render a prompt and add it, with its format, parser, and tools |
receive_image(image, caption=None) |
Add an image: a file path, a PIL image, or a base64 PNG, JPEG, WebP, or GIF |
respond(args=None, return_session=False) |
Run the agent loop on the active dialog |
await arespond(...) |
The same, for async code |
Responses
respond() returns the final Message. Its content is the text, and
parsed holds structured data when the prompt asked for a
format. Ask for the call session instead to
see how the answer was reached:
from pydantic import BaseModel
from lllm import Prompt, Tactic
class City(BaseModel):
name: str
country: str
ASK = Prompt(
path="geo/capital",
prompt="Name the capital of {country}.",
format=City,
)
agent = Tactic.quick(
system_prompt="You answer geography questions.", model="gpt-4o-mini"
)
agent.open("q")
agent.receive_prompt(ASK, {"country": "Japan"})
session = agent.respond(return_session=True)
print(session.state, session.delivery.parsed)
#> success {'name': 'Tokyo', 'country': 'Japan'}
print(session.exception_retries_count) # (1)!
#> 1
- The model's first reply did not match
City; the agent sent the error back, and the second reply parsed. One repair, no code on your side.
An AgentCallSession records every model call (invoke_traces), each
repair (exception_retries), each tool round (interrupts), provider
retries (llm_recalls), and the final delivery. Inside a tactic, every
agent call is also added to the tactic's session, with its cost.
The agent loop
respond() repeats until the model gives a final answer:
- Call the model with the dialog, the prompt's tools, and
model_args. - If the reply calls tools, run them, add the results, and go to 1.
- If the reply fails to parse (the prompt's parser or
format=), add the prompt's repair message (Error: ... Please fix.) and go to 1. - Otherwise add the reply to the dialog and return it.
Repairs and tool calls happen in a working copy, so the dialog you keep holds the clean exchange. Every branch has a budget:
| Setting | Default | Limits |
|---|---|---|
max_interrupt_steps |
5 | Tool rounds per call. After the last one the model is asked to answer without tools. 0 means up to 100. |
max_exception_retry |
3 | Repairs of unparsable replies, then AgentException is raised. |
max_llm_recall |
0 | Retries after a provider error. |
max_rate_limit_retries |
8 | Retries after rate limits, waiting min(60, 2**n) seconds plus jitter. |
Set them in the agent's config (or per agent in a tactic's
agent_configs). To change what the repair or tool-result messages say,
give the prompt a handler.
Model settings
model_args are sent with every call: temperature, max_tokens, seed,
a provider's api_base, and anything else LiteLLM accepts. Override them for
one call with respond(args=...):
from lllm import Tactic
agent = Tactic.quick(
system_prompt="You name things.", model="gpt-4o-mini", temperature=0.2
)
agent.open("names")
agent.receive("Suggest a name for a caching library.")
print(agent.respond(args={"temperature": 1.0}).content)
#> Stash
Models covers providers, local servers, and custom invokers, and Inference hints covers session-aware serving.
Async
arespond() runs the same loop without blocking the event loop, and inside a
tactic it is recorded like respond():
import asyncio
from lllm import Tactic
async def main():
agent = Tactic.quick(
system_prompt="You are terse.", model="gpt-4o-mini"
)
agent.open("chat")
agent.receive("What is 2 + 2?")
reply = await agent.arespond()
print(reply.content)
#> 4
asyncio.run(main())
Tools and skills
An agent's tools come from three places: the prompt driving the turn
(function_list), the agent's system prompt, and its config (tools:,
proxies, and skills). All of them stay available on every turn unless a
prompt sets agent_tools=False. See Tools.