Skip to content

Architecture Overview

LLLM organises an agentic system into four layers. Each layer has a single, clear responsibility and they compose cleanly.

┌──────────────────────────────────────────────────┐
│                    Tactic                        │  ← "the program"
│  (orchestrates agents, owns the task interface)  │
├────────────────┬────────────────┬────────────────┤
│    Agent A     │    Agent B     │    Agent C     │  ← "the callers"
│  (model+loop)  │  (model+loop)  │  (model+loop)  │
├────────────────┴────────────────┴────────────────┤
│               Prompts (functions)                │  ← "the calls"
│  template · parser · tools · handlers            │
├──────────────────────────────────────────────────┤
│               Dialogs (state)                    │  ← "mental state"
│  per-agent message history, fork-able tree       │
└──────────────────────────────────────────────────┘

The Four Abstractions

Tactic — the program

A Tactic is the top-level unit of an agentic system. It:

  • Accepts a task (string or Pydantic model) and returns a result
  • Owns a group of agents and wires them together
  • Is stateless — each tactic(task) call spins up fresh agent instances
  • Can be subclassed, shared, and reused like a library module
class Analytica(Tactic):
    name = "analytica"
    agent_group = ["analyzer", "synthesizer"]

    def call(self, task: str) -> str:
        ...

Agent — the caller

An Agent holds a system prompt and a model. It executes prompts through a call loop that handles:

  • Retries on LLM errors
  • Exception handling (parsing failures, bad output)
  • Interrupt handling (tool calls, multi-step reasoning)

An agent is not a long-running process — it operates on a Dialog (conversation state) that the tactic manages.

agent.open("dialog_alias")    # create/attach a dialog
agent.receive("user message") # append user turn
response = agent.respond()    # run the call loop, return Message

Prompt — the function

A Prompt is the specification for a single agent turn:

  • Template — string or .md file with {variable} slots
  • Parser — extracts structured output from the raw LLM text
  • Tools — regular function tools, tactic-backed tools, and proxy programming surfaces
  • Handlers — on_exception and on_interrupt prompts for the call loop

Prompts compose: one prompt can extend() another, inheriting its tools and parser.

Tool definitions have two regular programming styles:

  • Coupled declaration and implementation. @tool decorates a Python callable and produces a Function with both the schema and the implementation. Pass that object directly to Prompt(function_list=[...]), or register it in a package and reference it by URL such as shared_pkg.tools:search.
  • Decoupled declaration and implementation. A prompt can carry a bare Function declaration - effectively a header - while a package @tool(name=...) function provides the implementation. At runtime LLLM binds them by exact package key, so the prompt remains declarative and the implementation remains normal Python.

Tactic tools and proxies build on those ideas rather than forming unrelated tool systems. A tactic tool exposes a whole Tactic as a prompt-callable Function, which makes the abstraction graph recursive: a low-level prompt turn can call a high-level agentic subsystem. A proxy exposes an API surface through query_api_doc, run_python, and CALL_API, which is the programming-based path for agents that need many related endpoints or a small sandbox.

Dialog — the mental state

A Dialog is the per-agent conversation history. Key properties:

  • Append-only — messages are never mutated after appending
  • Fork-able — dialog.fork() creates a branch at any point, enabling parallel reasoning paths or exception-recovery sub-dialogs
  • Tree structure — forks form a tree; the agent always works on the active branch

Each agent maintains its own dialog. Agents don't share dialogs — they share information by passing content between them in the tactic's call() method.

Data Flow

tactic(task)
  │
  ├─► Agent A
  │     dialog.put_text(task)
  │     agent.respond()
  │       └─► agent call loop
  │             ├─► llm_invoker.call(dialog)   # API call
  │             ├─► parse output               # Prompt.parser
  │             ├─► handle tool calls          # Prompt.tools
  │             └─► return Message
  │
  └─► Agent B
        dialog.put_text(agent_a_result)
        agent.respond()
          └─► ...

Package System

The four abstractions describe what is inside an agentic system. The package system describes how it is organised and made available across files and projects.

A package is a folder with an lllm.toml manifest. It declares where your prompts, tools, configs, tactics, and proxies live, and LLLM handles discovery, namespacing, and lazy loading from there. All resources are addressable by a namespaced URL:

my_pkg.prompts:research/system
my_pkg.tools:search
my_pkg.proxies:market_data
my_pkg.tactics:research_writer

Multiple packages can declare dependencies on each other, enabling shared tactics and prompts to be imported like Python modules. This is what makes the transition from a single script to a production system clean — you never rewrite your agents or tactics, you just add structure around them.

See Package System for the full reference.

Design Philosophy

LLLM is designed for developers and researchers — in the spirit of PyTorch and Hugging Face: modular, composable, and easy to prototype with — so that agentic systems can be built, shared, and reused like ordinary software modules.

Core Principles

Agentic system as a program. An agentic system = agents (≈ system prompt + base model + proxies/skills, the "callers") + prompts (the "functions") + tactics (the "program" that wires them together). Treating each agent invocation as a well-defined function call — with explicit inputs, outputs, and error handling — minimises side effects, maximises compositionality, and makes parallelism straightforward.

Dialog as internal mental state. A dialog is the internal view each agent has of the conversation — not a shared global log. Different agents in a task maintain separate dialogs: each agent sees only what it has been told. The top_prompt at the head of the dialog also acts as the calling convention for the next turn, making the dialog a kind of function stack.

Configuration as declaration. System shape is declared in data (TOML/YAML), not hardcoded. What resources exist (prompts, proxies) and how they are wired (agent configs) are expressed as configuration — making systems inspectable, reproducible, and shareable without touching Python source.

Low-level by default. LLLM stops at Tactic as its highest abstraction. Higher-level orchestration — systems of systems, agent networks — is left to the application layer. For those patterns, see the SSSN framework.

Advanced Capabilities

Tool calling as programming. Beyond regular LLM tool calling, the proxy system wraps tools with rich metadata, documentation, and activation filtering — making tools composable and testable rather than just ad-hoc function schemas.

Tactics as a shared library. Tactics are reusable modules. Like a Python package, you can import a tactic from a shared library and drop it into your own system. Every prompt, proxy, config, and tactic is an independent, loadable resource — declared in lllm.toml and namespaced to avoid collisions.

Replayable logging. The logging system records every invocation with enough information to recreate the exact execution context of any run — prompts, model arguments, tool results, costs. This makes debugging, A/B testing, and prompt optimisation tractable on production traces.

Where to Go Next