{
  "access": "public",
  "type": "reference",
  "format": "markdown",
  "title": "Attention Cloud: Memory as a Rendering Problem",
  "url": "https://labs.datagrout.ai/papers/attention-cloud",
  "summary": "Every agent system confronts the same arithmetic: context windows are finite and history is not. The prevailing answer (append until overflow, then compact) treats the window as storage and compaction as a necessary loss. Both premises are wrong: the window is this frame's *view* of memory, and continuity is a rendering problem, not a storage problem.\n\nThe Attention Cloud is memory infrastructure built on that inversion. An agent's history is ingested deterministically into a durable journal of structured items, each annotated with the *goal* under which it was produced and carrying a stable pointer back to its verbatim source. Curation compiles the journal upward through levels of detail, verbatim to summary to distillation to skeleton, the way graphics pipelines compute mipmaps: one memory at multiple resolutions, the resolution chosen at read time. Hydration renders a frame: given a goal and a token budget, the platform assembles the relevant slice of memory at appropriate detail and returns it with a **receipt**, per-fact provenance stating what was included, why, at what cost, and by which retrieval path, including a declaration when retrieval degraded to recency.\n\nTwo further mechanisms make the system safe to run on itself. Injected memory is wrapped in a marked envelope so capture pipelines never re-ingest what recall injected. And because the detail ladder is both a compression scheme and a disclosure scale, sharing is a first-class operation: a grant exposes a segment of memory *at a ceiling of resolution* (full detail to a pairing partner, summaries to a team, skeleton to an organization), with receipts as the access log.",
  "topics": [
    "memory",
    "context-assembly",
    "level-of-detail",
    "provenance",
    "prolog"
  ],
  "content_markdown": "## Abstract\n\nEvery agent system confronts the same arithmetic: context windows are finite and history is not. The prevailing answer (append until overflow, then compact) treats the window as storage and compaction as a necessary loss. Both premises are wrong: the window is this frame's *view* of memory, and continuity is a rendering problem, not a storage problem.\n\nThe Attention Cloud is memory infrastructure built on that inversion. An agent's history is ingested deterministically into a durable journal of structured items, each annotated with the *goal* under which it was produced and carrying a stable pointer back to its verbatim source. Curation compiles the journal upward through levels of detail, verbatim to summary to distillation to skeleton, the way graphics pipelines compute mipmaps: one memory at multiple resolutions, the resolution chosen at read time. Hydration renders a frame: given a goal and a token budget, the platform assembles the relevant slice of memory at appropriate detail and returns it with a **receipt**, per-fact provenance stating what was included, why, at what cost, and by which retrieval path, including a declaration when retrieval degraded to recency.\n\nTwo further mechanisms make the system safe to run on itself. Injected memory is wrapped in a marked envelope so capture pipelines never re-ingest what recall injected. And because the detail ladder is both a compression scheme and a disclosure scale, sharing is a first-class operation: a grant exposes a segment of memory *at a ceiling of resolution* (full detail to a pairing partner, summaries to a team, skeleton to an organization), with receipts as the access log.\n\n---\n\n## Problem Landscape\n\n### Compaction Destroys What It Cannot Rank\n\nMid-session summarization must decide, irreversibly and in advance, what future turns will need. The failure is familiar to anyone who has watched an agent lose, at hour three, the constraint it was told at minute five — not because the constraint was unimportant, but because importance was judged before the question that needed it existed.\n\n### Retrieval Without Intent\n\nSystems that do persist history typically index it by content similarity. But an agent's memory is a record of *work*, not a corpus of documents, and the most useful retrieval key for work is not what the text resembles but what it was *for*. \"The last time we investigated this deploy pattern\" is a goal-shaped query that content similarity serves poorly, because the goal is mostly absent from the text it produced.\n\n### Recall Without Receipts\n\nWhen retrieved memory enters a context window unattributed, two failures follow. The agent cannot calibrate trust: a hazy recollection and a verbatim record look identical. And the human debugging the agent cannot answer the operative question: *why is this in the context?*\n\n### Memory That Eats Itself\n\nAny deployment where memory injection and history capture share a channel has a feedback loop: recalled content is captured as new content, curated into new memory, and recalled again, each cycle laundering provenance and amplifying whatever was recalled most.\n\n---\n\n## Design Principles\n\n### 1. The Journal Is Durable; the Window Is a Frame\n\nThe source of truth is a persistent journal of typed items in the agent's symbolic store. Sessions saturate a fixed window: the design target is a full window of relevant material assembled per turn, not a growing window periodically truncated. Nothing about the current frame constrains what the next frame may render.\n\n### 2. Memory Has Levels of Detail\n\nEach remembered item exists on a ladder: **L0**, the verbatim record, always paged back to its original source by a stable pointer; **L1**, a sentence-scale summary; **L2**, a structured distillation (entities, decisions, constraints, outcomes); **L3**, skeleton (tags, entities, and pointers only). Storage retains the ladder; readers choose the rung. Retention attaches per level, not per kind, and is set per installation: a constrained deployment can let raw records expire once their rollups exist, but the default keeps L0 long-term, because hydration renders a frame's hottest items back at full resolution and a discarded rung cannot be rendered. Skeletons persist indefinitely everywhere.\n\n```mermaid\nflowchart LR\n    L0[\"L0 · verbatim<br/>the full record, most tokens<br/>retention: configurable, long-term by default\"]\n    L1[\"L1 · summary<br/>one sentence<br/>retention: medium\"]\n    L2[\"L2 · distillation<br/>entities · decisions · constraints · outcomes<br/>retention: long\"]\n    L3[\"L3 · skeleton<br/>tags + entities + pointers, fewest tokens<br/>retention: indefinite\"]\n    L0 -- \"curation compiles upward\" --> L1 --> L2 --> L3\n    L3 -. \"a stable pointer pages back to the source\" .-> L0\n```\n\n*One item, four resolutions. The reader picks the rung at hydration time, and the same ladder doubles as the retention curve and the disclosure scale.*\n\n### 3. Goal at Write Time\n\nEvery ingested item carries the goal that was active when it was produced. This one annotation turns the journal from a transcript archive into a record of intentions and outcomes, and makes goal-shaped recall (\"when we last tried X\") a join rather than a guess.\n\n### 4. Every Recall Carries a Receipt\n\nHydration returns, alongside the assembled context, a per-fact accounting of what was included, from which segment, seeded by relevance or pulled in by expansion, at what estimated token cost, with an explicit marker when retrieval degraded to recency. The receipt serves the agent (calibration), the human (\"why is this here\"), and, in the sharing model below, the memory's *owner*.\n\n### 5. Injected Memory Is Marked\n\nEverything hydration injects is wrapped in a delimited envelope carrying its source and receipt reference. Capture pipelines strip envelopes before ingestion, so the journal records what the *agent and user* produced, never what memory injected. Verbatim echo is prevented by construction; paraphrased recycling is not, and is deferred to curation-time novelty controls.\n\n---\n\n## Architecture\n\n### Ingestion\n\nTranscripts become journal items through a deterministic parser: user goals, assistant conclusions (kept head-and-tail so endings survive truncation), file operations, and errors, with noise gated out. Item identity is a content-derived handle, so re-ingestion is an upsert and watching a live session is the same operation as backfilling an old one. When a session exceeds its item budget, each kind of item holds a fixed share, most-recent-first within kind; naive recency eviction was observed to discard exactly the assistant conclusions recall most needs.\n\nIngestion is a *client* of the platform, not a privileged path: multiple ingesters feed the same journal through the same interface, and enrollment is per-source and user-owned.\n\n### Curation\n\nA background curator compiles the ladder: consecutive same-goal items group into **arcs**; batched, inexpensive model passes produce arc summaries and tags and session rollups; the results are asserted back into the journal as facts, embedded for retrieval like everything else. Curation is the system's only routine inference spend, and its outputs are the L1/L2 rungs the render path prefers when budgets are tight.\n\n### Hydration\n\nA hydration request names a goal, a *plane* (which memory segments to draw from, each with its own token budget), and a total budget. Assembly seeds by semantic relevance against the journal's embeddings, expands through the entity graph, selects detail levels to fit the budget, and renders the frame with its receipt. Budgets are honored per segment, so one noisy namespace cannot crowd out the rest of the plane, and degradation is declared: when semantic seeding cannot run, the receipt says the frame was recency-seeded rather than pretending relevance.\n\nThe receipt is the answer to *why is this here*, per fact:\n\n```json\n{\n  \"goal\": \"the retry/backoff decision from the payments spike\",\n  \"budget\": { \"total\": 2000, \"spent\": 1840 },\n  \"path\": \"semantic\",\n  \"facts\": [\n    { \"handle\": \"arc_9f21\", \"segment\": \"incidents\", \"level\": \"L2\",\n      \"seeded_by\": \"relevance\", \"score\": 0.88, \"tokens\": 610 },\n    { \"handle\": \"msg_4c07\", \"segment\": \"incidents\", \"level\": \"L0\",\n      \"seeded_by\": \"entity_expansion\", \"score\": 0.71, \"tokens\": 900 },\n    { \"handle\": \"arc_1a3e\", \"segment\": \"decisions\", \"level\": \"L3\",\n      \"seeded_by\": \"relevance\", \"score\": 0.44, \"tokens\": 30 }\n  ]\n}\n```\n\n`path: \"semantic\"` degrades to `\"recency\"` when embeddings are unavailable.\n\n### Kinematic Traversal and Semantic Partitions\n\nAssembly is movement through memory, and it has two gaits, named by analogy with robot kinematics.\n\n**Inverse-kinematic (IK) jumps** land directly on the target. Given the goal, similarity search over the journal's embeddings returns seed items with no regard for what surrounds them: the fastest way to the most likely material, and structurally blind. When a segment has no embeddings to jump on, the jump degrades to time: the most recent items seed the frame, each marked as recency-sourced so the receipt can say so.\n\n**Forward-kinematic (FK) walks** expand from the seeds through structure. A jump teleports to the destination; the walk recovers what the jump skipped, the surrounding story. Today the walk follows the entity graph one hop: it pulls in facts whose subject or object is an entity named by a seed (the same session, the same file, the same named thing), seeds first and expansion after, until the segment's token budget or fact cap is reached.\n\nThe division of labor between stores follows the gaits. The relational store with its vector index is durability and the jump; the symbolic engine, which already holds the journal as facts, is where exhaustive walking belongs. `attention.nearest` is callable from inside a proof, so a rule can jump mid-inference and keep walking in Prolog.\n\nThe blend of gaits is **plane** policy, not code. An execution plane seeds narrowly and walks with a strong preference for staying on the trajectory; an exploration plane seeds widely and lets the walk run longer. The plane config exposes the seed count, the fact cap, and a token budget per segment today; the field weights that would tune the walk's stopping criterion are specified in *The Physics of Context*, not implemented.\n\n**Semantic Space Partitioning (SSP)** is the organization the walk moves through: memory divided into **cells**, neighborhoods of related items, with adjacency between cells and aggregates per cell (time range, entities, tags, and eventually a cell-level summary standing for its members). Two states:\n\n- *Dynamic cells today.* No cell structure is stored. The cell a frame draws from is computed at read time as the neighborhood the walk reached from its seeds, and adjacency is fact adjacency in the entity graph. This is enough to render a coherent frame; it is not enough to name a cell, mount it, or ask which cells are hot.\n- *Persisted cells next.* Cells become facts like everything else, `relation(Cell, contains, Handle)` and `relation(Cell, adjacent, Cell2)` in a dedicated `_at_cells` namespace, so a rule can navigate cell to cell with `call_tool/3` mid-inference without a round trip to the host, and a cell can be mounted as a scoped volume: one interstitial digging in one region of memory while another digs elsewhere, their rollups merging downstream.\n\n```mermaid\nflowchart LR\n    G[\"goal + plane<br/>segments, budgets, seed count\"]\n    IK[\"IK jump<br/>similarity lands on seed items<br/>(recency seeding as declared fallback)\"]\n    FK[\"FK walk<br/>expansion through structure:<br/>entity graph today, cell adjacency next\"]\n    C1[\"cell A<br/>the focus\"]\n    C2[\"cell B<br/>one hop out\"]\n    C3[\"cell C<br/>periphery\"]\n    LOD[\"LOD by distance from focus<br/>focus L0/L1 · one hop L2 · periphery L3<br/>(target; by order today)\"]\n    F[\"frame<br/>rendered within budget\"]\n    R[\"receipt<br/>per fact: seed or expansion,<br/>score, segment, rung, tokens\"]\n    G --> IK --> FK\n    FK --> C1\n    FK --> C2\n    FK --> C3\n    C1 --> LOD\n    C2 --> LOD\n    C3 --> LOD\n    LOD --> F\n    LOD --> R\n```\n\n*Two gaits and a partition. The jump finds the focus; the walk fills outward across cells; detail is assigned by distance from the focus; the frame and its receipt are rendered together.*\n\n**Foveation.** The frame is rendered the way a fovea sees: full resolution at the point of attention, falling off toward the edges, with the periphery present rather than cut off. Items at the focus (the seeds and their immediate arc) render at L0 or L1; items one hop out render at L2; the periphery renders at L3, tags and entities and the pointer that pages back to L0. The receipt records the rung chosen for every fact, so the shape of the fovea is inspectable afterward, and the boundary of the frame is visible from inside it: the agent can see that more memory exists past what it was given, and roughly where, without paying for it. The current allocator approximates the fovea by order rather than by demotion: seeds render first at their stored rung, the walk fills outward, and the budget truncates the tail; the outer reaches are lower-resolution because the journal already holds arcs (L1) and tags (L3) beside verbatim items. Demoting an item down the ladder to keep it present under pressure, and dithering skeleton breadcrumbs in from beyond the cutoff, are the allocator's next steps, specified but not implemented in *The Physics of Context*.\n\n### The Journal Is Facts\n\nThe journal lives in the platform's symbolic substrate (see *Logic Cells: A Persistent Prolog Substrate for Governed Agents*): items, arcs, and rollups are typed facts with relations, so memory composes with everything else facts compose with. Rules can traverse it, and investigations in the style of *Forensic Inference: Structured Agent Reasoning Over Accumulated Facts* can join memory against live observations.\n\n### Integration Points\n\nSeveral systems read and write this substrate, each a client of the journal through the same interfaces.\n\n- **Agentsmith interstitials** (*Agentsmith: A Symbolically-Grounded Agent Architecture*). Interstitials, the focused context-engineering sub-loops of a turn, acquire memory two ways. Before they run, the turn is hydrated: the journal segment through `attention.hydrate` on a small budget, and the working-memory segments (existing knowledge plus an internal checkpoint namespace) through the internal hydrator. During the turn, the memory-check interstitial queries the Logic Cell directly, with no model on the route. They write memory back by asserting: the execute interstitial dispatches assertions, queries, and constraints into Logic Cell namespaces, and every tool result is checkpointed as compact, hash-keyed facts the next hydration reads. Checkpoint facts carry no embeddings today, so the walk and recency seeding reach them, not the jump; the receipt says which.\n- **Manifold** (*Manifold: A Token-Efficient Coding Agent*). Manifold's navigation predicates over a codebase (callers, callees, neighbors, impact) run in the same Logic Cell that holds the journal, so code memory and session memory are one fact space, and the envelope's strip rule treats Manifold's injected context exactly as it treats recall. Cell-scoped navigation of code memory is what persisted partitions add.\n- **Lumen and the proxy ingesters.** Two ingesters feed the journal in one fact shape: a transcript watcher on machines running the CLI, and the Lumen proxy adapter, which maps each captured turn into journal items after stripping envelopes first and harness noise second, advancing a per-user watermark so re-ingestion is an upsert. One curator and one recall path serve both.\n\n---\n\n## Sharing: Resolution as Disclosure\n\nWhat follows is the design of the sharing model, not a description of a shipped implementation. The ladder built for compression is also the vocabulary for disclosure. A **grant** gives a grantee (a person, a team, a fleet) access to a **segment** of one's memory (a namespace filtered by project, tag, or time) *up to a maximum level of detail*. An L0 grant is pairing-partner intimacy: full resolution, typically time-boxed. An L2 grant exposes arcs and distillations, which is what a teammate needs. An L3 grant is presence: this person has memory about this territory; ask them.\n\nThe mechanics follow from treating memory as a governed resource:\n\n- **Mounts, not copies.** A grantee mounts a grant by reference, with their own token budget, and enforcement happens at read time, at the same choke point where budgets and detail selection already apply. Revocation is forward-only; receipts make what was already read auditable rather than pretending it unreadable.\n- **Receipts as the symmetric ledger.** The artifact that tells a reader why a fact is in their context tells the *owner* who read what, when, at which resolution. Sharing memory without an access log is not psychologically safe enough to use; the receipt is the log.\n- **Teams as first-class grantees.** Mutual pools are team policy (membership derives the grants, joining and leaving propagate automatically), never N² person-to-person grants. Pools are mountable by their members but not by other pools, which closes transitive laundering through organizational hierarchy.\n- **Derivatives inherit restriction monotonically.** Anything curation produces from mixed sources carries the most restrictive union of its sources' labels (readers intersect, restrictions union, retention takes the earliest date), so a summary cannot launder private items into shared memory through their own compression. Declassification exists only as an explicit, audited act.\n- **Erasure is staged, honest, and deep.** Forgetting an item tombstones it *and invalidates every derivative whose provenance includes it*, rebuilding those rollups without its influence while retaining a minimal audit receipt. Removing influence is supported; rewriting history is refused.\n\n---\n\n## Economics\n\nIngestion is deterministic and free of inference. Curation is the routine spend: batched small-model passes over new arcs, priced in the platform's ordinary credit terms. Hydration costs what its budget declares; the default posture is a few hundred tokens of garnish per recall. The structural comparison is against the default: replaying or re-summarizing history in-window pays tokens per *session* for work the journal pays once per *item*.\n\n---\n\n## Comparison with Existing Approaches\n\n| | Compaction | RAG over transcripts | Typed memory stores | Attention Cloud |\n|---|---|---|---|---|\n| Loss model | Irreversible, decided early | None, but flat | Schema decides | Layered; L0 always recoverable |\n| Retrieval key | N/A | Content similarity | Keys/slots | Goal + semantics + entity graph |\n| Detail control | One summary | Chunk size | Row shape | Per-read LOD selection |\n| Provenance | None | Chunk ref | Row ref | Per-fact receipt incl. retrieval path |\n| Echo safety | N/A | Unaddressed | Unaddressed | Marked envelope, strip-before-capture |\n| Sharing | N/A | Corpus-level | ACLs | Resolution-ceiling grants + access receipts |\n\n---\n\n## Limitations and Trade-Offs\n\n**Relevance is embedding-shaped today.** Richer selection pressure (novelty, tension with current work, interference between candidates) is specified in *The Physics of Context* and not implemented; the receipts record which path ran.\n\n**The ladder is populated unevenly.** L0 and arc-level summaries are routine; the full ladder as uniform, precomputed rungs across all content is the curator's ongoing work. The architecture treats missing rungs as degraded rendering, not failure.\n\n**The fovea is by order, not yet by demotion.** The periphery is lower-resolution because arcs and tags already exist beside verbatim items, not because the allocator demoted anything. Demotion-first allocation and dithering are the next steps at the assembly seam.\n\n**Invoked, not ambient.** Recall currently fires when asked, by an agent or a phrase-gated hook. The system does not yet notice on its own that memory would help.\n\n**Paraphrase recycling is unsolved.** The envelope stops verbatim echo; content a model restates in its own words re-enters the journal as apparently novel. Suppressing it belongs to curation-time near-duplicate detection and novelty discounting, not to delimiters.\n\n---\n\n## Future Directions\n\n### Pressure Fields\n\nThe intended selection layer treats context assembly as a field computation: relevance, novelty, coherence, tension, and interference as forces over the semantic space, with detail levels demoted under pressure rather than dropped. That design is developed in *The Physics of Context*; this paper's substrate (the ladder, the goal annotation, the receipts, the envelope) is the machine it would run on.\n\n### Persisted Semantic Partitions\n\nPersisting the partition into `_at_cells`, as described under Semantic Space Partitioning, adds what the dynamic form cannot offer: cell-level detail variants, so a hot cell's summary stands for its members at L2 the way an arc stands for its items; adjacency lookups in place of walks, so expansion reads a map instead of rediscovering it per frame; and cells a rule can navigate and mount.\n\n### Ambient Recall\n\nWith selection pressure worth trusting, recall can move from invoked to ambient: memory surfacing because the current goal's shape demands it, budgeted by the same planes and audited by the same receipts.\n\n---\n\n*This document describes the conceptual architecture of the Attention Cloud. Parsing rules, curation prompts, retrieval internals, and the sharing model's storage schema are part of the operational implementation and are not specified here.*\n",
  "last_updated": "2026-09-01T00:00:00Z"
}