Runtime & Registries

Most code only needs load_runtime()

Call load_runtime() once at startup to load your package, then refer to resources by name. Read on if you write tests with isolated registries, run parallel experiments with separate runtimes, or embed LLLM in a system that manages its own package loading.

A runtime is a registry: one store of ResourceNode objects keyed by namespaced URLs. It is where prompts, tools, proxies, tactics, configs, assets, and context managers are looked up by name.

from lllm import Runtime   # Runtime is an alias of Registry

Loading a package

Nothing is loaded when you import lllm. Populate the default runtime explicitly:

from lllm import load_runtime

runtime = load_runtime()   # finds lllm.toml, loads the package tree, sets it as default

Projects generated by lllm create my-app do this in main.py. Looking up a resource in a runtime that nothing has been loaded into raises a KeyError that says to call lllm.load_runtime().

load_runtime(toml_path=None, name=None, *, discover_cwd=True, discover_shared_packages=True):

Find the manifest. With toml_path, that file (or the lllm.toml in that folder). Otherwise $LLLM_CONFIG if set, then the first lllm.toml, .lllm.toml, or LLLM.toml in the working directory or its parents (a template/ subfolder at each level is checked too).
Load the package tree: the package, its dependencies (recursively, with cycle detection), and every resource section. See Package Discovery.
No manifest? With discover_cwd=True, the working directory is scanned for the standard folders (prompts/, tools/, proxies/, configs/, tactics/, assets/, services/). If any exist they are loaded as a package named after the directory, with a RuntimeWarning suggesting you add an lllm.toml. Otherwise the runtime stays empty. Pass discover_cwd=False for a strict, deterministic boot.
Shared packages. With discover_shared_packages=True, every sub-folder with an lllm.toml in <project root>/lllm_packages/ and ~/.lllm/packages/ is loaded too.
Store it. name=None makes the new runtime the default; a name stores it as a named runtime and leaves the default alone.

The default runtime

A process-wide default runtime exists from import time, empty and without side effects. Everything that takes runtime= / registry= falls back to it: Agent, Dialog, Tactic, ProxyManager, ProxyRegistrator, register_prompt, and the load_* helpers.

from lllm import get_default_runtime, set_default_runtime

runtime = get_default_runtime()
print(runtime.keys("prompt"))   # qualified keys of every prompt
print(runtime.default_namespace)  # root package name, used for bare keys

Named runtimes

For parallel experiments or tests, keep separate runtimes:

from lllm import get_runtime, load_runtime

load_runtime("experiment_a/lllm.toml", name="exp_a")
load_runtime("experiment_b/lllm.toml", name="exp_b")

rt_a = get_runtime("exp_a")
rt_b = get_runtime("exp_b")

tactic_a = MyTactic(config, registry=rt_a)

Each runtime has its own registry. get_runtime() with no name returns the default; an unknown name raises KeyError.

Internal architecture

All resources are ResourceNode objects in one _resources dict, keyed by qualified key ("my_pkg.prompts:greet/hello"). A _type_index maps resource types to keys for fast filtered iteration.

Registry
├── _resources: Dict[str, ResourceNode]    # "pkg.section:key" -> node
├── _type_index: Dict[str, Set[str]]       # "prompt" -> {"pkg.prompts:a", ...}
├── packages: Dict[str, PackageInfo]       # "pkg_name" (or alias) -> metadata
├── _loaded_package_paths: Set[str]        # cycle detection
└── _default_namespace: Optional[str]      # root package name

PackageInfo(name, version, description, base_dir, alias) describes a loaded package; effective_name is the alias if set.

Resource resolution

runtime.get(key, resource_type=None) and the typed helpers resolve keys in this order:

Exact match: key is a qualified key in the store.
Default namespace: for a key without :, try <default>.<section>:<key> for the requested type's section (prompt -> prompts, proxy -> proxies, ...), then each built-in section in order: tactics, services, configs, assets, prompts, tools, proxies.
Package shorthand: for pkg:key with no . in the package part and a requested type, try pkg.<section>:key.

So:

get_prompt("greet/hello") tries my_system.prompts:greet/hello
get_prompt("child_pkg:greet/hello") tries child_pkg.prompts:greet/hello
get("my_pkg.prompts:greet/hello") is an exact match

A miss raises KeyError listing the closest registered keys (or, in a runtime nothing has been loaded into, suggesting lllm.load_runtime()).

Registering resources

Through discovery

from lllm import load_package

load_package("path/to/lllm.toml", runtime=runtime)
# every prompt, tool, proxy, tactic, config, and asset is now registered

load_package(config_path=None, *, runtime=None, registry=None) loads one package tree into an existing runtime without the shared-package scan. The first package loaded into a runtime becomes its default namespace.

Typed methods

from lllm import Prompt, Runtime, tool

runtime = Runtime()

# Prompt: key is prompt.path
runtime.register_prompt(Prompt(path="test/greeting", prompt="Hello {name}!"), namespace="my_pkg.prompts")


# Tool: the key's last segment must equal the tool's name
@tool(description="Search documents.")
def search(query: str) -> str:
    return query


runtime.register_tool("search", search, namespace="my_pkg.tools")

# Proxy and tactic classes
runtime.register_proxy("search/web", WebProxy, namespace="my_pkg.proxies")
runtime.register_tactic("analytica", Analytica, namespace="my_pkg.tactics")

# Config: eager, or lazy with a loader
runtime.register_config("default", {"global": {"model_name": "gpt-4o"}}, namespace="my_pkg.configs")
runtime.register_config("heavy", loader=lambda: yaml.safe_load(open("heavy.yaml")),
                        namespace="my_pkg.configs")

# Context manager class: key is cls.name
runtime.register_context_manager(KeepLastTurns)

register_tool rejects a key whose last segment differs from tool.name; that exact-name rule is what binds prompt Function declarations to implementations. register_proxy and register_tactic default to overwrite=False and raise ValueError on a duplicate key; the other typed methods overwrite by default. Registering a prompt, tool, proxy, or tactic also records its qualified key and namespace on the object, which is how bare refs on a prompt resolve inside its own package.

Decorators that register at import time, such as @ProxyRegistrator without a runtime=, write to the default runtime.

Low-level API

from lllm.runtimes.native.core.resource import ResourceNode

node = ResourceNode.eager("logo.png", image_bytes, namespace="my_pkg.assets", resource_type="asset")
runtime.register(node)

# Lazy: the loader runs on first .value access, then the result is cached
node = ResourceNode.lazy("big_data", loader=lambda: load_parquet("data.parquet"),
                         namespace="my_pkg.datasets", resource_type="dataset")
runtime.register(node)

ResourceNode is a wrapper, not a base class: Prompt, Function, Tactic, and BaseProxy do not inherit from it. Extra keyword arguments to eager / lazy land in node.metadata (discovered files store file_path there). node.category is "platform" for tactics, services, configs, and assets, "native" for prompts, tools, proxies, and context managers, and "custom" otherwise.

Retrieving resources

# Typed (section inferred)
prompt = runtime.get_prompt("greet/hello")
tool_fn = runtime.get_tool("my_pkg:search")
tactic_cls = runtime.get_tactic("analytica")
proxy_cls = runtime.get_proxy("my_pkg.proxies:search/web")
config = runtime.get_config("default")          # triggers the lazy load
asset = runtime.get_asset("my_pkg:logo.png")
proxy_cls = runtime.get_proxy("my_pkg:search/web")  # package shorthand works for every type

# Generic
value = runtime.get("my_pkg.assets:logo.png")
node = runtime.get_node("my_pkg.assets:logo.png")
runtime.has("my_pkg.prompts:greet/hello")       # True / False

# Module-level helpers (default runtime unless runtime=/registry= is given)
from lllm import load_config, load_prompt, load_proxy, load_resource, load_tactic, load_tool
from lllm.runtimes.native.core.resource import load_asset, load_service_ref

prompt = load_prompt("child_pkg:greet/hello")
config = load_config("experiments/ablation", runtime=runtime)
asset = load_resource("my_pkg.assets:logo.png")

load_resource requires a : and accepts "pkg.section:key" or "section:key" (the section is looked up in the default package). A typed lookup that finds a resource of another type raises TypeError.

Isolated runtimes for testing

from lllm import Prompt, Runtime


def test_my_tactic():
    runtime = Runtime()
    runtime.register_prompt(Prompt(path="test/prompt", prompt="Hi {name}"), namespace="test.prompts")
    tactic = MyTactic(config, registry=runtime)
    result = tactic("hello")
    assert runtime.has("test.prompts:test/prompt")

Pair this with a scripted invoker (see Invokers) to test tactics without network access.

runtime.reset() clears every registry and the default namespace:

import pytest

from lllm import get_default_runtime


@pytest.fixture(autouse=True)
def clean_runtime():
    yield
    get_default_runtime().reset()

API reference

`Registry` / `Runtime`

Member	Description
`register(node, overwrite=True)`	Register any `ResourceNode`.
`get(key, resource_type=None)`	Resolve a key and return the node's value.
`get_node(key, resource_type=None)`	Resolve a key and return the `ResourceNode`.
`has(key)`	Whether a key resolves.
`keys(resource_type=None)`	Sorted qualified keys, optionally filtered by type.
`iter_items(resource_type=None)` / `iter_nodes(...)`	Deterministic iteration over `(key, node)` pairs / nodes.
`register_prompt(prompt, overwrite=True, namespace="")`	Register a prompt under `prompt.path`.
`get_prompt(path)`	Typed lookup.
`register_tool(name, tool, overwrite=True, namespace="")`	Register a `Function`; the key's last segment must equal `tool.name`.
`get_tool(path)`	Typed lookup.
`register_proxy(name, proxy_cls, overwrite=False, namespace="")`	Register a proxy class.
`get_proxy(path)`	Typed lookup.
`register_tactic(tactic_type, tactic_cls, overwrite=False, namespace="")`	Register a tactic class.
`get_tactic(name)`	Typed lookup.
`register_config(name, config_data=None, overwrite=True, namespace="", loader=None)`	Register a config, eager or lazy.
`get_config(path)`	Typed lookup (runs a lazy loader).
`get_asset(path)`	Typed lookup for `[assets]` files.
`register_service(name, service, overwrite=True, namespace="", loader=None)` / `get_service(path)`	Service resources (ASGI apps, service spec files).
`register_context_manager(cm_cls, overwrite=True, namespace="")`	Register a `ContextManager` class under `cm_cls.name`.
`get_context_manager(name)`	Typed lookup.
`register_package(pkg)`	Record a `PackageInfo`.
`packages`	Dict of loaded packages (aliases included).
`default_namespace`	Root package name used for bare keys.
`discovery_done`	`True` after `load_runtime` finished.
`loaded_package_paths`	Frozen set of loaded package folders.
`reset()`	Clear all state.

Module-level helpers

Importable from lllm (defined in lllm.runtimes.native.core.runtime):

Function	Description
`get_default_runtime()`	The process-wide default runtime.
`set_default_runtime(rt)`	Replace the default.
`get_runtime(name=None)`	A named runtime (`None` = default).
`load_runtime(toml_path=None, name=None, *, discover_cwd=True, discover_shared_packages=True)`	Create, populate, and store a runtime.

get_default_registry, set_default_registry, get_registry, and load_registry (in lllm.runtimes.native.core.runtime) are aliases.

Package management

Also importable from lllm; see Package Discovery for the workflow.

Function	Description
`install_package(zip_path, *, alias=None, scope="user", load=True, runtime=None)`	Install a package zip into `~/.lllm/packages/` (`scope="user"`) or `<project root>/lllm_packages/` (`scope="project"`). `alias` renames the package namespace; `load=False` skips loading it. Returns the installed folder.
`export_package(package_name, output_path, *, bundle_deps=False, runtime=None)`	Zip a loaded package (`.zip` is appended to the path if missing). `bundle_deps=True` packs its transitive dependencies under `lllm_packages/` inside the zip. Returns the zip path.
`list_packages(scope=None, *, runtime=None)`	Dicts with `name`, `version`, `description`, `scope`, `path` for installed packages, plus packages loaded in the runtime from elsewhere (`scope="unknown"`).
`remove_package(name, *, scope=None, runtime=None)`	Delete an installed package and unregister it. Raises `ValueError` if it exists in both scopes and no scope is given.

Design notes

Creating a runtime has no side effects. No scanning, no imports; it is an empty container until something registers into it.
One store, smart resolution. Each resource is stored once under its qualified key. Bare keys and package shorthand are resolved at read time, not stored as duplicates (aliases are the exception: an aliased dependency is registered again under its alias namespace).
Not thread-safe for concurrent registration. The normal pattern (load once at startup, read many times during agent calls) has no contention. If you register concurrently, use a lock or separate runtimes.