Native runtime
The objects you build agentic programs from. Import them from lllm (from lllm import Agent, Prompt, Tactic, tool), except TacticCallSession, AgentCallSession, BaseHandler, DefaultSimpleHandler, and tactic_as_function, which come from lllm.runtimes.native. Guides: Tactics, Agents, Prompts, Tools, Dialogs.
Tactics
Tactic
Tactic(config: Optional[Any] = None, runtime: Any = None, registry: Any = None, tactic_path: Optional[str] = None, context: Optional[TacticContext] = None, observer: Any = None, *, name: Optional[str] = None, description: Optional[str] = None, package_ref: Optional[str] = None, service_ref: Optional[str] = None, examples: Optional[List[Dict[str, Any]]] = None, metadata: Optional[Mapping[str, Any]] = None)
Bases: Tactic[InputT, OutputT]
The program of an agentic system.
A tactic owns a task interface and orchestrates the work behind it. Write
the program in :meth:call (call(self, task)); set agent_group and
pass agent configs to get fresh native :class:Agent objects in
self.agents on every call. Each call runs on its own copy of the
tactic inside a :class:TacticCallSession that records agent calls,
sub-tactic calls, and cost.
Tactic is also a protocol tactic: it validates typed input and output
(input_type/output_type, or the input_model/output_model
aliases), and exposes run/arun/info so it can be served,
streamed, packaged, and used as a tool by any runtime. Subclasses may
implement the protocol hook _run (or _arun) instead of call.
capabilities
capabilities() -> set[str]
Protocol surfaces plus the native call/acall entry points.
input_schema
input_schema() -> Any
Return the declared input schema/type, if any.
output_schema
output_schema() -> Any
Return the declared output schema/type, if any.
as_tool
as_tool(*, name: str | None = None, description: str | None = None, parameter_mode: str = 'task')
Expose this tactic as a typed Python callable.
Pydantic AI and other agent runtimes can consume ordinary Python callables as tools. The implementation is imported lazily so the core tactic protocol stays small and adapter-owned.
call
call(task: Any, **kwargs) -> Any
The tactic's program. Override this, or the protocol hook _run.
tactic(task), acall, bcall and ccall run call()
when a subclass writes it and its _run() (or _arun()) hook
otherwise, so both styles work through every entry point.
events
events(task: Any, **kwargs) -> Any
Return a sync event stream for tactics that support it.
aevents
async
aevents(input_value: Any, *, context: Any = None, **kwargs: Any) -> Any
Yield the events of an overridden :meth:events, else of astream().
astream
async
astream(input_value: Any, *, context: Any = None, **kwargs: Any) -> Any
Yield the items of an overridden stream(), run like :meth:call.
run
run(input_value: Any, *, context: Any = None, return_trace: bool = False, **kwargs: Any) -> Any
Run the tactic synchronously through the validated boundary.
A tactic that only implements _arun() runs arun() to
completion, so it has a sync entry point too.
quick
classmethod
quick(query: None = None, system_prompt: Optional[Union[str, Prompt]] = 'You are a helpful assistant.', model: str = 'gpt-4o', return_agent: bool = False, **model_args: Any) -> Agent
quick(query: Optional[str] = None, system_prompt: Optional[Union[str, Prompt]] = 'You are a helpful assistant.', model: str = 'gpt-4o', return_agent: bool = False, **model_args: Any) -> Union[Message, Agent, Tuple[Message, Agent]]
One agent, no configuration: Tactic.quick("What is LLLM?").
With query it returns the reply (and the agent when
return_agent=True); without query it returns the agent so you
can open/receive/respond yourself.
TacticCallSession
Bases: BaseModel
Tracks one invocation of a tactic — every agent call, every sub-tactic call, total cost, and the final result.
The tactic is stateless; all per-call data lives here.
Agents
Agent
dataclass
Agent(name: str, system_prompt: Prompt, model: str, llm_invoker: BaseInvoker, stream_handler: Optional[BaseStreamHandler] = None, runtime: Optional[Runtime] = None, api_type: APITypes = COMPLETION, model_args: Dict[str, Any] = dict(), max_exception_retry: int = 3, max_interrupt_steps: int = 5, max_llm_recall: int = 0, max_rate_limit_retries: int = 8, context_manager: Optional[ContextManager] = None, inference_hints: Optional[InferenceHints] = None, _dialogs: Dict[str, Dialog] = dict(), _active_alias: Optional[str] = None)
Represents a single LLM agent with a specific role and capabilities.
An Agent owns the dialogs it creates. Each dialog is keyed by a user-chosen alias (e.g. 'planning', 'talk_with_coder') that makes the code self-documenting:
agent.open('planning', prompt_args={...})
agent.receive("What's the plan?")
response = agent.respond()
agent.open('execution', prompt_args={...})
agent.switch('execution')
...
respond() runs the call loop on the active dialog. Every turn is
driven by the most recent prompt (its parser, format, and handlers); the
agent's own tools stay available on every turn unless that prompt sets
agent_tools=False.
Attributes:
| Name | Type | Description |
|---|---|---|
name |
str
|
The name or role of the agent (e.g., 'assistant', 'coder'). |
system_prompt |
Prompt
|
The system prompt defining the agent's persona. |
model |
str
|
The model identifier (e.g., 'gpt-4o'). |
llm_invoker |
BaseInvoker
|
The invoker instance for LLM calls. |
model_args |
Dict[str, Any]
|
Additional model arguments (temp, top_p, etc.). |
max_exception_retry |
int
|
Max retries for agent parsing/validation exceptions. |
max_interrupt_steps |
int
|
Max consecutive tool call interrupts. |
max_llm_recall |
int
|
Max retries for LLM API errors. |
dialogs
property
dialogs: Dict[str, Dialog]
Read-only snapshot of all managed dialogs (alias → Dialog).
open
open(alias: str, prompt_args: Optional[Dict[str, Any]] = None, session_name: Optional[str] = None, switch: bool = True)
Create a new dialog owned by this agent, keyed by alias.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
alias
|
str
|
the alias for the new dialog. |
required |
prompt_args
|
Optional[Dict[str, Any]]
|
the arguments for the system prompt. |
None
|
session_name
|
Optional[str]
|
the name of the session for logging and checkpointing. |
None
|
switch
|
bool
|
if True, switch to the new dialog after opening. Default is True. |
True
|
fork
fork(alias: str, child_alias: str, last_n: int = 0, first_k: int = 1, switch: bool = True) -> Agent
Branch an existing dialog into a new child dialog.
The parent dialog's fork() handles all lineage bookkeeping
(parent ↔ child links, split_point, ids). Agent just stores
the child under child_alias and switches to it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
alias
|
str
|
the source dialog to fork from. |
required |
child_alias
|
str
|
the alias for the new child dialog. |
required |
last_n
|
int
|
if >0, the child keeps only the last n messages (plus the first k); 0 copies the whole dialog. |
0
|
first_k
|
int
|
when last_n > 0, also keep the first k messages (usually the system prompt). |
1
|
switch
|
bool
|
if True, switch to the new child dialog after forking. |
True
|
Raises:
| Type | Description |
|---|---|
ValueError
|
if |
KeyError
|
if |
close
close(alias: str) -> Dialog
Remove a dialog from this agent and return it.
Useful for archiving, handing off to another system, or just cleaning up. If the closed dialog was active, active becomes None.
switch
switch(alias: str) -> Agent
Set the active dialog by alias. Returns self for chaining.
Raises:
| Type | Description |
|---|---|
KeyError
|
if |
receive
receive(text: str, alias: Optional[str] = None, role: Roles = USER, name: str = 'user') -> Message
Put a text message into the active (or specified) dialog.
receive_prompt
receive_prompt(prompt: Prompt, prompt_args: Optional[Dict[str, Any]] = None, alias: Optional[str] = None, role: Roles = USER, name: str = 'user') -> Message
Put a structured prompt message into the dialog.
receive_image
receive_image(image, caption: Optional[str] = None, alias: Optional[str] = None, role: Roles = USER, name: str = 'user') -> Message
Put an image message into the dialog.
arespond
async
arespond(alias: Optional[str] = None, metadata: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, parser_args: Optional[Dict[str, Any]] = None, return_session: bool = False) -> Union[Message, AgentCallSession]
respond() for async code: runs the call off the event loop.
respond
respond(alias: Optional[str] = None, metadata: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, parser_args: Optional[Dict[str, Any]] = None, return_session: Literal[False] = False) -> Message
respond(alias: Optional[str] = None, metadata: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, parser_args: Optional[Dict[str, Any]] = None, return_session: Literal[True] = True) -> AgentCallSession
respond(alias: Optional[str] = None, metadata: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, parser_args: Optional[Dict[str, Any]] = None, return_session: bool = False) -> Union[Message, AgentCallSession]
High-level: run the agent call loop on a dialog, return the response.
This is the recommended way to get a response. For full diagnostics
(call_state with retry info, model_args, etc.), use call() directly.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
alias
|
Optional[str]
|
the alias of the dialog to respond to. |
None
|
metadata
|
Optional[Dict[str, Any]]
|
additional metadata for the call. |
None
|
args
|
Optional[Dict[str, Any]]
|
additional arguments for the prompt. |
None
|
parser_args
|
Optional[Dict[str, Any]]
|
arguments for the output parser. |
None
|
return_session
|
bool
|
if True, return the entire AgentCallSession instead of just the message (use session.delivery to get the final message). |
False
|
AgentCallSession
Bases: BaseModel
Tracking the state of the agent call, including the exceptions, interrupts, and LLM recalls.
It can be helpful for debugging, analysis, and writing custom handlers. Like error-wise hints, interrupt-wise hints, etc.
tool_calls
property
tool_calls: List[FunctionCall]
Every tool call the agent ran during this call, in order.
InferenceHints
Bases: BaseModel
Serving hints an agent attaches to each model call.
priority follows the Dynamo/SGLang convention: higher values are more
urgent. The vllm profile converts it to vLLM's convention, where lower
values are scheduled first.
session_ids
session_ids(dialog_id: Optional[str], parent_dialog_id: Optional[str] = None) -> tuple[Optional[str], Optional[str]]
Session and parent-session ids for a dialog (or (None, None)).
request_fields
request_fields(*, dialog_id: Optional[str] = None, parent_dialog_id: Optional[str] = None, model_args: Optional[Dict[str, Any]] = None) -> Dict[str, Dict[str, Any]]
Return {"extra_headers": ..., "extra_body": ...} for one call.
The result plugs into LiteLLM's completion(**kwargs); empty parts
are omitted.
final_request_fields
final_request_fields(dialog_id: str) -> Dict[str, Dict[str, Any]]
Fields for Dynamo's end-of-session request (x-dynamo-session-final).
Send them on a dedicated minimal request when a dialog is finished so lifecycle-aware routers can release per-session state immediately.
Prompts
Prompt
Bases: BaseModel
A Prompt is a complete behaviour definition for one agent turn.
It bundles four concerns:
- Template — the text to send to the LLM, with
{variable}placeholders rendered viastr.format(or a customrenderer). - Output contract — an :class:
OutputSpecdescribing how to parse and validate the LLM's response. - Tools — the :class:
Functionand :class:MCPobjects available during this turn. - Handlers — template strings (or full Prompts) that define how to recover from exceptions and how to feed tool results back.
Provider-specific features (web search, computer use, citations, …) live
in the generic capabilities dict so that new features never require
schema changes on Prompt.
Notes
The __call__ method uses str.format by default, so literal braces
in the template must be doubled: {{ and }}.
template_vars
property
template_vars: set[str]
Variable names required by this template (e.g. {topic} → 'topic').
validate_args
validate_args(prompt_args: dict[str, Any]) -> list[str]
Return list of missing required template variables for the prompt.
get_function
get_function(name: str) -> Function
Retrieve a declared Function by name, with a clear error.
resolve_function_refs
resolve_function_refs(runtime: Optional[Runtime] = None) -> 'Prompt'
Resolve tool declarations into executable Functions.
String entries may be regular tool resources or tactic tool resources.
Unimplemented Function objects are declarations that bind to a
regular tool resource with the same key in the prompt's package.
Resolution is package-local first when the prompt has a qualified
resource key.
extend
extend(**overrides: Any) -> Prompt
Create a new Prompt inheriting all fields, with overrides applied.
A new path is required — prompts that share a path would collide
in the registry::
child = parent.extend(
path="child/analysis",
prompt="More specific: {task}",
output=OutputSpec(parser=strict_parser),
)
info_dict
info_dict() -> Dict[str, Any]
Return a JSON-serializable snapshot suitable for experiment tracking.
to_dict
to_dict() -> Dict[str, Any]
A JSON-safe snapshot (tool callables and parsers are not included).
Tools are stored as their schemas (implementations are re-bound by
name when the prompt is resolved against a runtime); a Pydantic
format model is stored as its import path and JSON schema.
BaseParser
Minimal parser interface.
DefaultTagParser
ParseError
Bases: ValueError
Raised when a parser cannot satisfy its declared contract.
BaseHandler
Bases: ABC
Base class for all handlers.
You may use call_state to build more complex handlers, like rule-based, event-driven, or even an agentic handler like bug fixing agent.
DefaultSimpleHandler
Bases: BaseHandler, BaseModel
A simple handler for the prompt.
User may override the exception message, interrupt message, and interrupt final message to build a more complex handler.
Tools
tool
tool(description: Optional[str] = None, prop_desc: Optional[Mapping[str, str]] = None, *, name: Optional[str] = None, strict: Optional[bool] = None, processor: Callable = _default_function_call_processor) -> Callable[[Callable], Function]
Decorator that turns a plain Python function into a :class:Function.
Usage::
@tool(description="Get current weather for a city")
def get_weather(location: str, units: str = "celsius") -> str:
return "Sunny, 22°C"
# get_weather is now a Function instance with its implementation.
prompt = Prompt(path="bot", prompt="...", function_list=[get_weather])
Function
Bases: BaseModel
Declarative description of a callable tool.
The schema (name, description, properties, required) describes the tool
to the LLM. The implementation is supplied by a package tool resource
with the same key, usually created by the :func:tool decorator.
from_callable
classmethod
from_callable(fn: Callable, *, name: Optional[str] = None, description: Optional[str] = None, prop_desc: Optional[Mapping[str, str]] = None, strict: Optional[bool] = None, processor: Callable = _default_function_call_processor) -> Function
Build a :class:Function by inspecting fn's signature and
docstring. Type hints are converted to JSON Schema types; nested
models are inlined, and the models of a recursive cycle are spelled
out once each, with other references back into the cycle described
by type and title ({"type": "object", "title": "Node"}).
Parameters without a type annotation default to "string".
Parameters whose names end with * in the docstring (or that
lack defaults) are treated as required.
For example:
@tool(
description="Get the current weather in a given location"
prop_desc={
"location": "The city and state, e.g. San Francisco, CA",
"unit": "The unit of temperature, e.g. celsius, fahrenheit",
}
)
def get_weather(location: str, unit: str = "celsius") -> str:
... # whatever you want to return, be sure to return a string at the end
FunctionCall
Bases: BaseModel
One invocation of a tool, including its result once executed.
tactic_as_function
tactic_as_function(tactic: Any, *, name: str | None = None, description: str | None = None, parameter_mode: Literal['task', 'kwargs'] = 'task', strict: Optional[bool] = None, processor: Callable = _default_function_call_processor) -> Function
Expose an LLLM tactic as a native prompt Function.
The tool's parameters come from the tactic's published input schema:
one task parameter holding the whole input (parameter_mode="task")
or the input model's fields as separate parameters ("kwargs").
An input without fields (e.g. str) cannot be split into keyword
arguments, so "kwargs" falls back to the single task parameter.
Nested and recursive models are inlined as in
:meth:Function.from_callable.
MCP
Bases: BaseModel
Dialogs
Dialog
dataclass
Dialog(session_name: Optional[str] = None, top_prompt: Optional[Prompt] = None, runtime: Optional[Runtime] = None, owner: Optional[str] = None, _messages: List[Message] = list(), tree_node: Optional[DialogTreeNode] = None, _parent_dialog: Optional['Dialog'] = None, _children_dialogs: List['Dialog'] = list())
An append-only message sequence owned by a single agent.
The agent that creates a dialog seeds it with its system prompt, and that
ownership is recorded on the tree_node. Other participants (user,
tools, forwarded messages from other agents) append via put_* helpers,
but the system-level identity of the dialog never changes.
Tree structure is maintained by a :class:DialogTreeNode owned by
each Dialog. fork() creates a child Dialog whose tree_node is
automatically linked to the parent's tree_node — callers (including
Agent) never need to wire lineage manually.
parent
property
parent: Optional['Dialog']
Live reference to parent Dialog (None for root dialogs).
children
property
children: List['Dialog']
Live references to child Dialogs forked from this one.
put_image
put_image(image: Union[str, Path, Any], caption: Optional[str] = None, name: str = 'user', metadata: Optional[Dict[str, Any]] = None, role: Roles = USER) -> Message
Expects: - image: a base64 encoded string, a Path object or string path, or a PIL Image object
fork
fork(last_n: int = 0, first_k: int = 1) -> 'Dialog'
Create a child dialog branching from this one.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
last_n
|
int
|
if >0, only preserve the last n messages from the parent dialog (useful for retrying from an earlier point). The first system message is always preserved. |
0
|
first_k
|
int
|
if >0, ensure the first k messages from the parent dialog (useful for preserving the system prompt message). Should always be >=1 to at least preserve the system prompt message. |
1
|
The fork automatically: - Deep-copies the message prefix into the child. - Creates a child DialogTreeNode linked to this dialog's tree_node. - Records split_point on the child's tree_node. - Wires live Dialog-level parent/children refs. - Inherits session_name, top_prompt, runtime, owner.
Returns:
| Type | Description |
|---|---|
'Dialog'
|
The new child Dialog. |
tree_overview
tree_overview(indent: int = 0) -> str
Recursively print the dialog tree structure from this node. Useful for debugging multi-fork scenarios.
Message
Bases: BaseModel
Configs and packages
resolve_config
resolve_config(name: str, runtime: Optional[Runtime] = None, *, registry: Optional[Registry] = None, _visited: Optional[set] = None) -> Dict[str, Any]
Load a config by name and resolve base inheritance.
The base key points to another config name (no .yaml suffix).
Inheritance is recursive — each level's keys override the parent's,
with dict values merged deeply (so model_args from both parent
and child are combined, not replaced wholesale).
Parameters
name:
Config resource name, e.g. "default" or
"agent_cfgs/agent1" or "my_pkg:default".
runtime:
Compatibility name for the registry to look up configs from.
registry:
Registry to look up configs from.
Returns
The fully merged config dict (base key removed).
build_tactic
build_tactic(config: Any, name: Optional[str] = None, runtime: Optional['Runtime'] = None, *, registry: Optional['Registry'] = None, **kwargs) -> 'Tactic'
Build a Tactic from a config/profile dict.
load_runtime
load_runtime(toml_path: Optional[str] = None, name: Optional[str] = None, *, discover_cwd: bool = True, discover_shared_packages: bool = True) -> Runtime
Create and populate a Runtime from a TOML file.
name=None → replaces the default runtime. name="something" → stored as a named runtime.
Parameters
discover_cwd:
When True and no lllm.toml is found, auto-discover resource
folders in the current working directory. Set to False for
deterministic "strict boot" startup.
discover_shared_packages:
When True, auto-load packages found in standard shared-package
locations. Set to False when the caller wants full control over
which packages enter the runtime.
load_prompt
load_prompt(path: str, runtime: 'Optional[Runtime]' = None, *, registry: 'Optional[Runtime]' = None) -> 'Prompt'
Load a prompt. Accepts "resource", "pkg:resource",
or "pkg.prompts:resource".
install_package
install_package(zip_path: 'str | Path', *, alias: Optional[str] = None, scope: str = 'user', load: bool = True, runtime: Optional[Runtime] = None, registry: Optional[Registry] = None) -> 'Path'
Install an LLLM package from a zip file.
Extracts zip_path, reads the package name from the top-level
lllm.toml, and copies the package to the appropriate shared-packages
directory.
Parameters
zip_path:
Path to the .zip file produced by :func:export_package or any
zip archive with a top-level folder containing an lllm.toml.
alias:
Install the package under this name instead of the name declared in
its lllm.toml. The folder is renamed and the name field in
lllm.toml is updated so the namespace prefix changes too. Use
this when the package's original name collides with something already
installed.
scope:
"user" (default) — installs to ~/.lllm/packages/, available
across all projects for the current user.
"project" — installs to <project_root>/lllm_packages/,
committed with the repo and shared with the team.
load:
If True (default) immediately loads the installed package into
runtime so it is available without restarting Python.
runtime:
Runtime to load into. Defaults to the current default runtime.
Returns
Path The installed package directory.
Raises
FileNotFoundError
If zip_path does not exist.
ValueError
If the zip contains no recognisable package or the zip structure is
ambiguous.
FileExistsError
If a package with the same name is already installed. Pass
alias=<new_name> to install under a different namespace.
Examples
Basic install::
from lllm import install_package
install_package("finance-toolkit-v1.2.zip")
# Resources now accessible as finance-toolkit:...
Install with alias to avoid a name collision::
install_package("finance-toolkit-v1.2.zip", alias="ft")
# Resources accessible as ft:...
Install into the project (committed to repo)::
install_package("finance-toolkit-v1.2.zip", scope="project")