Skip to content

Native runtime

The objects you build agentic programs from. Import them from lllm (from lllm import Agent, Prompt, Tactic, tool), except TacticCallSession, AgentCallSession, BaseHandler, DefaultSimpleHandler, and tactic_as_function, which come from lllm.runtimes.native. Guides: Tactics, Agents, Prompts, Tools, Dialogs.

Tactics

Tactic

Tactic(config: Optional[Any] = None, runtime: Any = None, registry: Any = None, tactic_path: Optional[str] = None, context: Optional[TacticContext] = None, observer: Any = None, *, name: Optional[str] = None, description: Optional[str] = None, package_ref: Optional[str] = None, service_ref: Optional[str] = None, examples: Optional[List[Dict[str, Any]]] = None, metadata: Optional[Mapping[str, Any]] = None)

Bases: Tactic[InputT, OutputT]

The program of an agentic system.

A tactic owns a task interface and orchestrates the work behind it. Write the program in :meth:call (call(self, task)); set agent_group and pass agent configs to get fresh native :class:Agent objects in self.agents on every call. Each call runs on its own copy of the tactic inside a :class:TacticCallSession that records agent calls, sub-tactic calls, and cost.

Tactic is also a protocol tactic: it validates typed input and output (input_type/output_type, or the input_model/output_model aliases), and exposes run/arun/info so it can be served, streamed, packaged, and used as a tool by any runtime. Subclasses may implement the protocol hook _run (or _arun) instead of call.

capabilities

capabilities() -> set[str]

Protocol surfaces plus the native call/acall entry points.

input_schema

input_schema() -> Any

Return the declared input schema/type, if any.

output_schema

output_schema() -> Any

Return the declared output schema/type, if any.

as_tool

as_tool(*, name: str | None = None, description: str | None = None, parameter_mode: str = 'task')

Expose this tactic as a typed Python callable.

Pydantic AI and other agent runtimes can consume ordinary Python callables as tools. The implementation is imported lazily so the core tactic protocol stays small and adapter-owned.

call

call(task: Any, **kwargs) -> Any

The tactic's program. Override this, or the protocol hook _run.

tactic(task), acall, bcall and ccall run call() when a subclass writes it and its _run() (or _arun()) hook otherwise, so both styles work through every entry point.

events

events(task: Any, **kwargs) -> Any

Return a sync event stream for tactics that support it.

aevents async

aevents(input_value: Any, *, context: Any = None, **kwargs: Any) -> Any

Yield the events of an overridden :meth:events, else of astream().

astream async

astream(input_value: Any, *, context: Any = None, **kwargs: Any) -> Any

Yield the items of an overridden stream(), run like :meth:call.

run

run(input_value: Any, *, context: Any = None, return_trace: bool = False, **kwargs: Any) -> Any

Run the tactic synchronously through the validated boundary.

A tactic that only implements _arun() runs arun() to completion, so it has a sync entry point too.

quick classmethod

quick(query: None = None, system_prompt: Optional[Union[str, Prompt]] = 'You are a helpful assistant.', model: str = 'gpt-4o', return_agent: bool = False, **model_args: Any) -> Agent
quick(query: str, system_prompt: Optional[Union[str, Prompt]] = 'You are a helpful assistant.', model: str = 'gpt-4o', return_agent: Literal[False] = False, **model_args: Any) -> Message
quick(query: str, system_prompt: Optional[Union[str, Prompt]] = 'You are a helpful assistant.', model: str = 'gpt-4o', return_agent: Literal[True] = True, **model_args: Any) -> Tuple[Message, Agent]
quick(query: Optional[str] = None, system_prompt: Optional[Union[str, Prompt]] = 'You are a helpful assistant.', model: str = 'gpt-4o', return_agent: bool = False, **model_args: Any) -> Union[Message, Agent, Tuple[Message, Agent]]

One agent, no configuration: Tactic.quick("What is LLLM?").

With query it returns the reply (and the agent when return_agent=True); without query it returns the agent so you can open/receive/respond yourself.

TacticCallSession

Bases: BaseModel

Tracks one invocation of a tactic — every agent call, every sub-tactic call, total cost, and the final result.

The tactic is stateless; all per-call data lives here.

Agents

Agent dataclass

Agent(name: str, system_prompt: Prompt, model: str, llm_invoker: BaseInvoker, stream_handler: Optional[BaseStreamHandler] = None, runtime: Optional[Runtime] = None, api_type: APITypes = COMPLETION, model_args: Dict[str, Any] = dict(), max_exception_retry: int = 3, max_interrupt_steps: int = 5, max_llm_recall: int = 0, max_rate_limit_retries: int = 8, context_manager: Optional[ContextManager] = None, inference_hints: Optional[InferenceHints] = None, _dialogs: Dict[str, Dialog] = dict(), _active_alias: Optional[str] = None)

Represents a single LLM agent with a specific role and capabilities.

An Agent owns the dialogs it creates. Each dialog is keyed by a user-chosen alias (e.g. 'planning', 'talk_with_coder') that makes the code self-documenting:

agent.open('planning', prompt_args={...})
agent.receive("What's the plan?")
response = agent.respond()

agent.open('execution', prompt_args={...})
agent.switch('execution')
...

respond() runs the call loop on the active dialog. Every turn is driven by the most recent prompt (its parser, format, and handlers); the agent's own tools stay available on every turn unless that prompt sets agent_tools=False.

Attributes:

Name Type Description
name str

The name or role of the agent (e.g., 'assistant', 'coder').

system_prompt Prompt

The system prompt defining the agent's persona.

model str

The model identifier (e.g., 'gpt-4o').

llm_invoker BaseInvoker

The invoker instance for LLM calls.

model_args Dict[str, Any]

Additional model arguments (temp, top_p, etc.).

max_exception_retry int

Max retries for agent parsing/validation exceptions.

max_interrupt_steps int

Max consecutive tool call interrupts.

max_llm_recall int

Max retries for LLM API errors.

current_dialog property

current_dialog: Dialog

The currently active dialog.

dialogs property

dialogs: Dict[str, Dialog]

Read-only snapshot of all managed dialogs (alias → Dialog).

open

open(alias: str, prompt_args: Optional[Dict[str, Any]] = None, session_name: Optional[str] = None, switch: bool = True)

Create a new dialog owned by this agent, keyed by alias.

Parameters:

Name Type Description Default
alias str

the alias for the new dialog.

required
prompt_args Optional[Dict[str, Any]]

the arguments for the system prompt.

None
session_name Optional[str]

the name of the session for logging and checkpointing.

None
switch bool

if True, switch to the new dialog after opening. Default is True.

True

fork

fork(alias: str, child_alias: str, last_n: int = 0, first_k: int = 1, switch: bool = True) -> Agent

Branch an existing dialog into a new child dialog.

The parent dialog's fork() handles all lineage bookkeeping (parent ↔ child links, split_point, ids). Agent just stores the child under child_alias and switches to it.

Parameters:

Name Type Description Default
alias str

the source dialog to fork from.

required
child_alias str

the alias for the new child dialog.

required
last_n int

if >0, the child keeps only the last n messages (plus the first k); 0 copies the whole dialog.

0
first_k int

when last_n > 0, also keep the first k messages (usually the system prompt).

1
switch bool

if True, switch to the new child dialog after forking.

True

Raises:

Type Description
ValueError

if child_alias is already in use.

KeyError

if alias doesn't exist.

close

close(alias: str) -> Dialog

Remove a dialog from this agent and return it.

Useful for archiving, handing off to another system, or just cleaning up. If the closed dialog was active, active becomes None.

switch

switch(alias: str) -> Agent

Set the active dialog by alias. Returns self for chaining.

Raises:

Type Description
KeyError

if alias doesn't exist.

receive

receive(text: str, alias: Optional[str] = None, role: Roles = USER, name: str = 'user') -> Message

Put a text message into the active (or specified) dialog.

receive_prompt

receive_prompt(prompt: Prompt, prompt_args: Optional[Dict[str, Any]] = None, alias: Optional[str] = None, role: Roles = USER, name: str = 'user') -> Message

Put a structured prompt message into the dialog.

receive_image

receive_image(image, caption: Optional[str] = None, alias: Optional[str] = None, role: Roles = USER, name: str = 'user') -> Message

Put an image message into the dialog.

arespond async

arespond(alias: Optional[str] = None, metadata: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, parser_args: Optional[Dict[str, Any]] = None, return_session: bool = False) -> Union[Message, AgentCallSession]

respond() for async code: runs the call off the event loop.

respond

respond(alias: Optional[str] = None, metadata: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, parser_args: Optional[Dict[str, Any]] = None, return_session: Literal[False] = False) -> Message
respond(alias: Optional[str] = None, metadata: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, parser_args: Optional[Dict[str, Any]] = None, return_session: Literal[True] = True) -> AgentCallSession
respond(alias: Optional[str] = None, metadata: Optional[Dict[str, Any]] = None, args: Optional[Dict[str, Any]] = None, parser_args: Optional[Dict[str, Any]] = None, return_session: bool = False) -> Union[Message, AgentCallSession]

High-level: run the agent call loop on a dialog, return the response.

This is the recommended way to get a response. For full diagnostics (call_state with retry info, model_args, etc.), use call() directly.

Parameters:

Name Type Description Default
alias Optional[str]

the alias of the dialog to respond to.

None
metadata Optional[Dict[str, Any]]

additional metadata for the call.

None
args Optional[Dict[str, Any]]

additional arguments for the prompt.

None
parser_args Optional[Dict[str, Any]]

arguments for the output parser.

None
return_session bool

if True, return the entire AgentCallSession instead of just the message (use session.delivery to get the final message).

False

AgentCallSession

Bases: BaseModel

Tracking the state of the agent call, including the exceptions, interrupts, and LLM recalls.

It can be helpful for debugging, analysis, and writing custom handlers. Like error-wise hints, interrupt-wise hints, etc.

tool_calls property

tool_calls: List[FunctionCall]

Every tool call the agent ran during this call, in order.

InferenceHints

Bases: BaseModel

Serving hints an agent attaches to each model call.

priority follows the Dynamo/SGLang convention: higher values are more urgent. The vllm profile converts it to vLLM's convention, where lower values are scheduled first.

session_ids

session_ids(dialog_id: Optional[str], parent_dialog_id: Optional[str] = None) -> tuple[Optional[str], Optional[str]]

Session and parent-session ids for a dialog (or (None, None)).

request_fields

request_fields(*, dialog_id: Optional[str] = None, parent_dialog_id: Optional[str] = None, model_args: Optional[Dict[str, Any]] = None) -> Dict[str, Dict[str, Any]]

Return {"extra_headers": ..., "extra_body": ...} for one call.

The result plugs into LiteLLM's completion(**kwargs); empty parts are omitted.

final_request_fields

final_request_fields(dialog_id: str) -> Dict[str, Dict[str, Any]]

Fields for Dynamo's end-of-session request (x-dynamo-session-final).

Send them on a dedicated minimal request when a dialog is finished so lifecycle-aware routers can release per-session state immediately.

Prompts

Prompt

Bases: BaseModel

A Prompt is a complete behaviour definition for one agent turn.

It bundles four concerns:

  1. Template — the text to send to the LLM, with {variable} placeholders rendered via str.format (or a custom renderer).
  2. Output contract — an :class:OutputSpec describing how to parse and validate the LLM's response.
  3. Tools — the :class:Function and :class:MCP objects available during this turn.
  4. Handlers — template strings (or full Prompts) that define how to recover from exceptions and how to feed tool results back.

Provider-specific features (web search, computer use, citations, …) live in the generic capabilities dict so that new features never require schema changes on Prompt.

Notes

The __call__ method uses str.format by default, so literal braces in the template must be doubled: {{ and }}.

template_vars property

template_vars: set[str]

Variable names required by this template (e.g. {topic} → 'topic').

validate_args

validate_args(prompt_args: dict[str, Any]) -> list[str]

Return list of missing required template variables for the prompt.

get_function

get_function(name: str) -> Function

Retrieve a declared Function by name, with a clear error.

resolve_function_refs

resolve_function_refs(runtime: Optional[Runtime] = None) -> 'Prompt'

Resolve tool declarations into executable Functions.

String entries may be regular tool resources or tactic tool resources. Unimplemented Function objects are declarations that bind to a regular tool resource with the same key in the prompt's package. Resolution is package-local first when the prompt has a qualified resource key.

extend

extend(**overrides: Any) -> Prompt

Create a new Prompt inheriting all fields, with overrides applied.

A new path is required — prompts that share a path would collide in the registry::

child = parent.extend(
    path="child/analysis",
    prompt="More specific: {task}",
    output=OutputSpec(parser=strict_parser),
)

info_dict

info_dict() -> Dict[str, Any]

Return a JSON-serializable snapshot suitable for experiment tracking.

to_dict

to_dict() -> Dict[str, Any]

A JSON-safe snapshot (tool callables and parsers are not included).

Tools are stored as their schemas (implementations are re-bound by name when the prompt is resolved against a runtime); a Pydantic format model is stored as its import path and JSON schema.

BaseParser

Minimal parser interface.

DefaultTagParser

Bases: BaseParser, BaseModel

Extract XML blocks, fenced markdown blocks, and signal tags.

ParseError

Bases: ValueError

Raised when a parser cannot satisfy its declared contract.

BaseHandler

Bases: ABC

Base class for all handlers.

You may use call_state to build more complex handlers, like rule-based, event-driven, or even an agentic handler like bug fixing agent.

DefaultSimpleHandler

Bases: BaseHandler, BaseModel

A simple handler for the prompt.

User may override the exception message, interrupt message, and interrupt final message to build a more complex handler.

Tools

tool

tool(description: Optional[str] = None, prop_desc: Optional[Mapping[str, str]] = None, *, name: Optional[str] = None, strict: Optional[bool] = None, processor: Callable = _default_function_call_processor) -> Callable[[Callable], Function]

Decorator that turns a plain Python function into a :class:Function.

Usage::

@tool(description="Get current weather for a city")
def get_weather(location: str, units: str = "celsius") -> str:
    return "Sunny, 22°C"

# get_weather is now a Function instance with its implementation.
prompt = Prompt(path="bot", prompt="...", function_list=[get_weather])

Function

Bases: BaseModel

Declarative description of a callable tool.

The schema (name, description, properties, required) describes the tool to the LLM. The implementation is supplied by a package tool resource with the same key, usually created by the :func:tool decorator.

from_callable classmethod

from_callable(fn: Callable, *, name: Optional[str] = None, description: Optional[str] = None, prop_desc: Optional[Mapping[str, str]] = None, strict: Optional[bool] = None, processor: Callable = _default_function_call_processor) -> Function

Build a :class:Function by inspecting fn's signature and docstring. Type hints are converted to JSON Schema types; nested models are inlined, and the models of a recursive cycle are spelled out once each, with other references back into the cycle described by type and title ({"type": "object", "title": "Node"}).

Parameters without a type annotation default to "string". Parameters whose names end with * in the docstring (or that lack defaults) are treated as required.

For example:

@tool(
    description="Get the current weather in a given location"
    prop_desc={
        "location": "The city and state, e.g. San Francisco, CA",
        "unit": "The unit of temperature, e.g. celsius, fahrenheit",
    }
)
def get_weather(location: str, unit: str = "celsius") -> str:
    ... # whatever you want to return, be sure to return a string at the end

FunctionCall

Bases: BaseModel

One invocation of a tool, including its result once executed.

tactic_as_function

tactic_as_function(tactic: Any, *, name: str | None = None, description: str | None = None, parameter_mode: Literal['task', 'kwargs'] = 'task', strict: Optional[bool] = None, processor: Callable = _default_function_call_processor) -> Function

Expose an LLLM tactic as a native prompt Function.

The tool's parameters come from the tactic's published input schema: one task parameter holding the whole input (parameter_mode="task") or the input model's fields as separate parameters ("kwargs"). An input without fields (e.g. str) cannot be split into keyword arguments, so "kwargs" falls back to the single task parameter. Nested and recursive models are inlined as in :meth:Function.from_callable.

MCP

Bases: BaseModel

Dialogs

Dialog dataclass

Dialog(session_name: Optional[str] = None, top_prompt: Optional[Prompt] = None, runtime: Optional[Runtime] = None, owner: Optional[str] = None, _messages: List[Message] = list(), tree_node: Optional[DialogTreeNode] = None, _parent_dialog: Optional['Dialog'] = None, _children_dialogs: List['Dialog'] = list())

An append-only message sequence owned by a single agent.

The agent that creates a dialog seeds it with its system prompt, and that ownership is recorded on the tree_node. Other participants (user, tools, forwarded messages from other agents) append via put_* helpers, but the system-level identity of the dialog never changes.

Tree structure is maintained by a :class:DialogTreeNode owned by each Dialog. fork() creates a child Dialog whose tree_node is automatically linked to the parent's tree_node — callers (including Agent) never need to wire lineage manually.

parent property

parent: Optional['Dialog']

Live reference to parent Dialog (None for root dialogs).

children property

children: List['Dialog']

Live references to child Dialogs forked from this one.

put_image

put_image(image: Union[str, Path, Any], caption: Optional[str] = None, name: str = 'user', metadata: Optional[Dict[str, Any]] = None, role: Roles = USER) -> Message

Expects: - image: a base64 encoded string, a Path object or string path, or a PIL Image object

fork

fork(last_n: int = 0, first_k: int = 1) -> 'Dialog'

Create a child dialog branching from this one.

Parameters:

Name Type Description Default
last_n int

if >0, only preserve the last n messages from the parent dialog (useful for retrying from an earlier point). The first system message is always preserved.

0
first_k int

if >0, ensure the first k messages from the parent dialog (useful for preserving the system prompt message). Should always be >=1 to at least preserve the system prompt message.

1

The fork automatically: - Deep-copies the message prefix into the child. - Creates a child DialogTreeNode linked to this dialog's tree_node. - Records split_point on the child's tree_node. - Wires live Dialog-level parent/children refs. - Inherits session_name, top_prompt, runtime, owner.

Returns:

Type Description
'Dialog'

The new child Dialog.

tree_overview

tree_overview(indent: int = 0) -> str

Recursively print the dialog tree structure from this node. Useful for debugging multi-fork scenarios.

Message

Bases: BaseModel

Configs and packages

resolve_config

resolve_config(name: str, runtime: Optional[Runtime] = None, *, registry: Optional[Registry] = None, _visited: Optional[set] = None) -> Dict[str, Any]

Load a config by name and resolve base inheritance.

The base key points to another config name (no .yaml suffix). Inheritance is recursive — each level's keys override the parent's, with dict values merged deeply (so model_args from both parent and child are combined, not replaced wholesale).

Parameters

name: Config resource name, e.g. "default" or "agent_cfgs/agent1" or "my_pkg:default". runtime: Compatibility name for the registry to look up configs from. registry: Registry to look up configs from.

Returns

The fully merged config dict (base key removed).

build_tactic

build_tactic(config: Any, name: Optional[str] = None, runtime: Optional['Runtime'] = None, *, registry: Optional['Registry'] = None, **kwargs) -> 'Tactic'

Build a Tactic from a config/profile dict.

load_runtime

load_runtime(toml_path: Optional[str] = None, name: Optional[str] = None, *, discover_cwd: bool = True, discover_shared_packages: bool = True) -> Runtime

Create and populate a Runtime from a TOML file.

name=None → replaces the default runtime. name="something" → stored as a named runtime.

Parameters

discover_cwd: When True and no lllm.toml is found, auto-discover resource folders in the current working directory. Set to False for deterministic "strict boot" startup. discover_shared_packages: When True, auto-load packages found in standard shared-package locations. Set to False when the caller wants full control over which packages enter the runtime.

load_prompt

load_prompt(path: str, runtime: 'Optional[Runtime]' = None, *, registry: 'Optional[Runtime]' = None) -> 'Prompt'

Load a prompt. Accepts "resource", "pkg:resource", or "pkg.prompts:resource".

install_package

install_package(zip_path: 'str | Path', *, alias: Optional[str] = None, scope: str = 'user', load: bool = True, runtime: Optional[Runtime] = None, registry: Optional[Registry] = None) -> 'Path'

Install an LLLM package from a zip file.

Extracts zip_path, reads the package name from the top-level lllm.toml, and copies the package to the appropriate shared-packages directory.

Parameters

zip_path: Path to the .zip file produced by :func:export_package or any zip archive with a top-level folder containing an lllm.toml. alias: Install the package under this name instead of the name declared in its lllm.toml. The folder is renamed and the name field in lllm.toml is updated so the namespace prefix changes too. Use this when the package's original name collides with something already installed. scope: "user" (default) — installs to ~/.lllm/packages/, available across all projects for the current user. "project" — installs to <project_root>/lllm_packages/, committed with the repo and shared with the team. load: If True (default) immediately loads the installed package into runtime so it is available without restarting Python. runtime: Runtime to load into. Defaults to the current default runtime.

Returns

Path The installed package directory.

Raises

FileNotFoundError If zip_path does not exist. ValueError If the zip contains no recognisable package or the zip structure is ambiguous. FileExistsError If a package with the same name is already installed. Pass alias=<new_name> to install under a different namespace.

Examples

Basic install::

from lllm import install_package
install_package("finance-toolkit-v1.2.zip")
# Resources now accessible as finance-toolkit:...

Install with alias to avoid a name collision::

install_package("finance-toolkit-v1.2.zip", alias="ft")
# Resources accessible as ft:...

Install into the project (committed to repo)::

install_package("finance-toolkit-v1.2.zip", scope="project")