Loom

Type to search across the guides, specification and decision records.

↑↓ navigate ↵ open esc close Pagefind

Decision records

ADR 0016 — Loom 2.0: Execution Runtime + the Decision seam

  • Status: Accepted (Milestones 1–11 + AI-native DX + kernel-hardening K1–K5 implemented — Loom 2.0 roadmap complete)
  • Date: 2026-07-08

Context

A “Loom 2.0” vision proposes reframing Loom from a durable workflow runtime into an execution runtime for intelligent software — 12 “runtimes” (execution, scheduler, reasoning, memory, capability, compute, projection, agent, event-intelligence, self-optimization, AI-native DX, runtime-intelligence). Exploring the repo showed most of these already exist as real-but-skeletal packages, so 2.0 is a re-layering and deepening of the principled core (docs/architecture.md, P1–P8, ADRs 0001–0015), not a rewrite.

A critical evaluation found the vision’s great instinct — everything is an execution over a durable log, with pluggable seams — worth keeping, but three recurring failure modes worth correcting:

  1. Over-runtime-ification. Making all 12 concerns mandatory layers bloats the kernel and taxes the simple case. Kernels win by staying small.
  2. Re-inventing durability. “Everything executes as fibers” with fiber-level pause/resume/checkpoint/migration collides with the event-sourced suspend/resume that already owns durable pause/resume (@loom/scheduler even documents this) — two sources of truth for “where am I.”
  3. Non-determinism smuggled in as features. Branch prediction, speculative execution, preemption, and self-optimization map badly onto coarse, expensive, side-effecting LLM work and several silently violate P2 (pure deterministic replay).

Decision

Two new principles (extend P1–P8; never regress).

  • P9 — Non-determinism is allowed, but only when logged. A runtime may route, elevate, learn, or optimize, but every such choice is emitted as a durable DECISION_RECORDED event. Replay stays a pure fold (P2 intact); a given run is reproducible forever; outcomes still improve run-to-run.
  • P10 — Minimal kernel, advisory plane. The durable core stays small; all “intelligence” is an optional module in an ephemeral advisory plane that affects outcomes only via logged decisions (P9) or latency only via off-log warming — never a hidden input to the fold.

Two spines everything hangs off.

  • Spine A — the Execution seam (M1, implemented here). Generalize the durable core from workflows of steps to executions of tasks; a step is one task kind. WorkflowRuntime→ExecutionRuntime (alias kept), WorkflowContext→ExecutionContext (alias kept). New ctx.execute(task, input) is the generalized primitive; ctx.run(step, input) becomes a thin alias that forwards a StepDefinition (the ‘step’ task). The PCB stays minimal (Execution: id/status/frames/sequence/lineage/tags); scheduling, reasoning, memory, and compute concerns live in a separate, optional ExecutionPolicy referenced by id — never embedded in the core type. The wire events stay STEP_*, so existing logs replay byte-identically and the fold needs no change.
  • Spine B — the Decision seam (M2, implemented here). A ctx.decide(id, () => optimizer()) primitive and a DECISION_RECORDED event (folded into a reserved _decisions map): on first run the optimizer is consulted and its choice recorded; on replay/resume the recorded choice is served (via ReplayVM.hasDecision/getDecision) and the optimizer is never called — the exact twin of how ctx.run short-circuits completed steps. This one primitive is the substrate for reasoning routing/elevation (P3), predictive recommendations (P9), self-optimization (P10), and runtime intelligence (P12).

Per-phase disposition (keep / change / drop). Keep: execution seam, reasoning routing (P3, highest value), capability deepening (P5), compute generalization (P6), projection surfaces (P7), thinner agents (P8), read-only event analytics (P9). Change: scheduler “speculation” → latency-only warming; self-optimization → opt-in/auditable/logged; NL→app → plain-static-TypeScript codegen only; memory recall that affects outcomes must be logged. Drop: fibers as a durable primitive, speculative task execution, true preemption. Push out of the kernel to AetherOS: marketplace, knowledge-graph/org-memory.

Refined, risk-aware ordering. M1 Execution seam → M2 Decision seam → { M3 Reasoning ‖ M3.5 scale-out PostgresEventStore/SnapshotStore/BranchRegistry + cross-node subscribe ‖ M4 compute placement ‖ M5 memory tiers } → M6 scheduler in hot path → M7 capability → M8 agent re-base → M9–M11 intelligence (last, optional). Every milestone ships behind optional config and defaults to today’s behavior (P: incrementally adoptable). The full roadmap, including tech-stack scalability analysis, lives in the planning doc referenced below.

Consequences

  • Additive and behavior-preserving. ctx.run is unchanged for callers; execute(step) and run(step) emit an identical event stream (regression-tested). All prior names are still exported from @loom/core, @loom/workflow-runtime, and @loom/sdk.
  • Kernel stays small. Reasoning/memory/compute metadata is out-of-core (ExecutionPolicy), preserving P7 (pure core) / P8 (pluggable edges).
  • Determinism preserved and extended. P2 is untouched; P9 gives non-determinism a legal, logged home so later “smart” phases can’t silently break replay.
  • AI-native DX — NL → codegen (P11, reshaped). @loom/appgen turns a natural-language intent into a structured AppSpec via a deterministic, offline directive parser (parseIntent), then emits plain, static, inspectable TypeScript you own (generateApp) — flows/steps/capabilities
    • a defineConfig root, with reasoning/approval/UI wired only when asked. There is no runtime interpretation of NL: the sole “understanding” step is the (swappable) parser producing a spec; codegen is pure and template-based, so the output is ordinary statically-wired Loom code (P4/P5), not a cage. Dep-free; a CLI/editor decides where to write the returned files.
  • Event intelligence, self-optimization & runtime intelligence (M9–M11/P9–P12). @loom/analytics provides pure, read-only analyzers over the durable log — execution / cost / reasoning / failure (“lenses on the log”, D11) — reading the facts M1–K3 already write (RESOURCE_METERED, DECISION_RECORDED, EFFECT_DENIED, QUOTA_EXCEEDED). rankModels mines historical routes into a ranking that a new rankedRouter (@loom/reasoning) applies to prefer proven models — an explicit, opt-in optimizer whose choices are logged via the decision seam (P9), auditable and override-able, never silently autonomous (the reshaped P10/P12). runtimeIntelligence composes analytics + optimizers into an advisory report (rankings, hot failures, a budget suggestion) that shapes future runs but never mutates the fold — the advisory plane in one function.
  • Scheduler in the hot path (M6/P2). EffectRuntime optionally routes every effect through a @loom/scheduler Scheduler (via EffectRuntimeOptions.scheduler) for bounded concurrency + priority ordering across a runtime’s effects (admission control); priority comes from EffectContext.priority. An optional Warmer pre-warms a model/connection/cache off the log (latency-only, no event — P10 advisory plane). Zero-overhead by default: with neither configured, execute is the unchanged direct call, and the scheduler’s Infinity-concurrency fast path is preserved.
  • Capability deepening (M7/P5). CapabilityDefinition gains version and dependencies; CapabilityCatalog.resolve(id) returns transitive deps in dependency-first order (detecting missing deps and cycles); and an optional CapabilityInvokeAuthorizer on the catalog denies unauthorized invokes (distinct from the K2 effect ring — this governs who may invoke a capability). All additive; discovery/routing for existing catalogs is unchanged.
  • Agents re-based on the runtimes (M8/P8). AgentRunner now accepts an optional ReasoningRuntime (with provider made optional; one is required). When supplied, the agent’s model calls are routed/elevated by the Reasoning Runtime and the chosen model is recorded as a durable DECISION_RECORDED fact (source: "agent", P9) — model heavy-lifting moves out of the agent into the runtime, so agents get thinner. The bare-provider path is byte-for-byte unchanged (existing agents, e.g. agent-swarm, keep their behaviour).
  • Memory tiers landed (M4/P4 → M5). @loom/memory gains TieredMemory over the scoped MemoryRuntime: named tiers (DEFAULT_TIERS = working/episodic/semantic/procedural), each with an eviction (ttlMs/maxEntries) or summarization (summarizeAfter, pluggable Summarizer, auto-run on write) policy, plus promotion between tiers. Critically, recall accepts a decide sink (pass ctx.decide): the recalled set is then recorded durably (P9), so memory-driven outcomes replay deterministically even if the store later changes — the fix that keeps non-deterministic recall from breaking P2. Opt-in and additive; MemoryRuntime is unchanged.
  • Reasoning Runtime (M3) landed on the decision seam. @loom/reasoning provides a ReasoningRuntime built as composable stages — a model-route registry, a pluggable Router (cheap-first tagRouter), a pluggable ConfidenceSignal (stop-reason / validator, not a magic self-report), and a confidence-based elevation ladder (Gemma→…→Opus). Each route and elevation is recorded through an optional DecideFn (shaped like ctx.decide), so routing is observable and replay-safe (P9). @loom/intent and @loom/planner were re-based to prefer it, keeping their offline keyword/heuristic fallbacks deterministic. It is an optional advisory module — the kernel is untouched (P10).
  • Fault supervision landed (K4 — kernel-hardening). @loom/dispatcher gains a SupervisingDispatcher that wraps any child dispatcher with an explicit restart/backoff policy and a per-key circuit breaker (open after threshold consecutive dispatch failures, fail fast with CircuitOpenError until a cooldownMs half-open trial) — so a flaky model/tool/worker isn’t hammered. Composes over LocalDispatcher/PlacementDispatcher unchanged; cancel/health delegate.
  • Event-schema ABI governance landed (K5 — kernel-hardening). The event log is the kernel’s on-disk ABI. @loom/replay-vm exposes EVENT_SCHEMA_VERSION (semver) + knownEventTypes(), and a governance test pins the handled event-type set so a removal/rename (which changes how historical logs fold) can’t happen silently — the on-disk twin of K1’s syscall pin, backing the flow-level guard already provided by @loom/deploy-sim.
  • Kernel ABI formalized (K1 — kernel-hardening). ExecutionContext is declared the stable, versioned “syscall” surface between userspace (flows/agents) and the kernel: KERNEL_ABI_VERSION (semver) + a KERNEL_SYSCALLS table (execute, run, decide, parallel, suspend, emit, stream, ui) with a compatibility guarantee (add ⇒ minor; remove/rename ⇒ major). A conformance test pins the surface against a live context so the ABI cannot drift silently. Exported from @loom/workflow-runtime (and @loom/sdk).
  • Resource accounting & quotas landed (K3 — kernel-hardening). The scarce resource for intelligent software is tokens / $ / latency. EffectContext gains a running resources snapshot (threaded by WorkflowContext from the durable fold). meteringMiddleware(pricing, budget?) (@loom/middleware) meters each effect as a durable RESOURCE_METERED fact (folded into variables._resources) and, given a budget, enforces a quota by admission control — once spent, the next effect is denied (QUOTA_EXCEEDED + QuotaExceededError, failing the step). Replay-safe: metered facts live in the log, the running total is read from the deterministic fold. Opt-in; no metering middleware ⇒ unchanged.
  • Protection ring landed (K2 — kernel-hardening). The EffectRuntime — the single chokepoint every side effect flows through — is now an opt-in capability-checked authority boundary. An EffectRuntime given a CapabilityAuthorizer authorizes each invocation; an ungranted effect never runs, emits a durable EFFECT_DENIED audit fact (folded into variables._denials, P9), and throws CapabilityDeniedError (failing the step). GrantTable is the least-privilege authorizer: a static principal→effect-ids map ("*" wildcard, pluggable principal for per-execution grants), wired via defineConfig({ authorizer }). No authorizer ⇒ today’s permissive behaviour, so existing apps are unchanged. This is the one boundary a library can enforce, and it is what lets a platform be safely built on the kernel. See roadmap Part 10.
  • Compute placement policy landed (M4). @loom/dispatcher gains a PlacementDispatcher — a composite ExecutionDispatcher that routes each dispatch to one of several named target dispatchers (local / remote-HTTP / Cloud Run) via a pure PlacementPolicy, driven by the M1 ExecutionPolicy through dispatchMetadataFromPolicy (computeTarget→target, priority, resources→estimatedCpu/Memory, affinity→lane). Each target is itself a dispatcher, so retry/timeout/lease handling is reused and no cloud knowledge leaks into the router. With one target it behaves exactly like that target, so existing single-placement dispatch and LocalDispatcher are unchanged. cancel routes to the handling target; health aggregates.
  • Scalability unblocker landed (M3.5). @loom/store-postgres provides PostgresEventStore
    • PostgresSnapshotStore + PostgresBranchRegistry behind the same interfaces as the SQLite stores (JSONB payloads, copyPrefix server-side via ||, BIGINT seq/ts). Live subscribe works in-process synchronously and across nodes via LISTEN/NOTIFY (each store ignores its own NOTIFYs to avoid double-delivery). It is drop-in via defineConfig’s new { kind: "postgres", url } (lazy-loaded like sqlite, so pg never reaches browser bundles) or the existing custom kind. Single-writer-per-execution (D3) stays the lock-free shard model — sequences are still allocated in-process, so append needs no read-then-write, and UNIQUE(execution_id, sequence) catches any accidental double-writer. Verified against a live Postgres 17 (15 tests incl. cross-node subscribe and byte-identical replay vs SQLite), gated on LOOM_TEST_POSTGRES_URL so CI without a database skips cleanly.

Verification

  • pnpm typecheck clean; pnpm test = 90/90 turbo tasks pass, including ExecuteParity (M1: run(step) and execute(step) produce an identical event stream; execute accepts a raw TaskDefinition routed by toolId), DecisionSeam (M2: ctx.decide records once, resume serves the recorded choice without re-invoking the optimizer), and the @loom/reasoning suite plus intent-through-reasoning integration tests (M3: cheap-first routing, confidence-based elevation, decision logging). The @loom/store-postgres suite (M3.5) runs against a live Postgres when LOOM_TEST_POSTGRES_URL is set and skips otherwise.
  • pnpm bench within committed baselines (scheduler fast path 1.01×; no fold/render/scheduler regression) — the P10 “zero-overhead default” gate holds.
  • Existing workflow-runtime, replay-vm, deploy-sim, and eval suites pass unchanged, confirming stored logs still replay.

See also

  • Planning roadmap: ~/.claude/plans/i-plan-to-evolve-mossy-bumblebee.md (critical evaluation, per-phase verdicts, tech-stack scalability assessment, per-milestone success criteria).
  • Principles: docs/architecture.md §2 (P9, P10 added).

Source: docs/adr/0016-loom-2.0-execution-runtime-and-decision-seam.md in the loom repository.