Paper: Understanding Is Getting the Context Right — An OS for Language Models

[MikeyBeez]

Sharing a design paper on context management as an operating system concern — relevant to Semantic Kernel's approach to orchestrating LLM interactions and managing conversation state.

Core argument: Context selection, not context length, is the dominant factor in reasoning quality. The paper proposes a two-agent architecture:

• A curator agent that continuously manages what enters the reasoning agent's context window — analogous to a kernel managing memory for user-space processes
• Threaded conversation history (DAG) instead of flat sequential logs
• Two manifests per turn: a topic index for scope awareness, and a curated active payload
• Provenance tracking — distinguishing user-stated facts from tool outputs from model inferences
• Exponential decay with current theory marking — controlled forgetting with user-overridable persistence
• A persistent repository that accumulates into a knowledge graph across sessions

The OS metaphor runs deep: context management is memory management, protocols are the instruction set, and the whole system learns through use without retraining weights.

Paper and PDF: github.com/MikeyBeez/fuzzyOS
DOI: 10.5281/zenodo.18571717

Thoughts welcome from people working on kernel-level orchestration for LLMs.

[darfaz]

Interesting paper — the OS metaphor resonates. A few thoughts from the security angle:

Provenance tracking is underrated for security. The distinction between user-stated facts, tool outputs, and model inferences maps directly to trust levels. In a real OS, you don't let user-space data execute as kernel code. Similarly, retrieved documents (tool outputs) shouldn't be trusted at the same level as system instructions — that's exactly how indirect prompt injection works.

The curator agent concept could enforce this: before context enters the reasoning agent's window, validate that retrieved content doesn't contain injected instructions. This is essentially an input firewall at the context management layer.

The exponential decay model has a security implication too. If poisoned context persists across sessions (via the knowledge graph), you get memory poisoning — an attack vector where adversarial content planted in one session influences future behavior. The "user-overridable persistence" needs to be carefully gated.

For anyone implementing these patterns, scanning context at ingestion time helps catch these issues early. ClawMoat is one option — it detects prompt injection and data exfiltration attempts in text payloads, which maps well to validating what the curator agent admits into the active context.

Would be curious if the paper addresses adversarial robustness of the curator agent itself — who watches the watchman?

[MikeyBeez]

Good points!