Everyone is obsessed with giving AI a “memory,” but we’re mostly just building better filing cabinets. We talk about Vectors for semantic vibes, Graphs for relationships, and SQL for the cold, hard facts. Some of us—myself included—even lean on a tiered filesystem (the Viking Context Protocol) because sometimes a flat Markdown file is the only thing that doesn’t hallucinate. But the truth is that none of these are actually “memory.” They are just retrieval systems with a marketing budget.
The Failure of Retrieval-as-Memory
The core hangup of current tiered systems is the Retrieval Gap. You can have a 100TB Qdrant index, but if the agent doesn’t know to ask the right question, that data might as well not exist. Vectors are great for finding “things that look like this,” but they are terrible at temporal nuance. Graphs are brilliant for deep relationships, but they brittle out as the nodes grow. And SQL? SQL is where context goes to die in a normalized table.
We keep trying to mirror human memory—episodic, semantic, procedural—but LLMs aren’t human. They don’t “remember” in the background; they only exist in the moment they are being prompted. Every memory system we build today is an active, tool-based retrieval. It requires the agent to consciously reach out and pull context. That’s not memory; that’s research.
The Passive Hydration Dream
To move beyond this, we need to stop thinking about “retrieval” and start thinking about Passive Hydration. Imagine a BMitM (Brain-in-the-Middle) proxy—a middleware layer that sits between the agent and the world. Instead of the agent asking, “What did Jason say about coffee last week?”, the proxy hydrates the inbound context with relevant facts before the agent even sees it. It’s a digital subconscious.
Passive hydration solves the “analysis paralysis” of decision branching. If the context is already there—if the agent is “wet” with relevant history—the reasoning becomes fluid. We don’t need a bigger context window; we need a smarter filter for what goes into that window.
The Subconscious Layer
Current LLMs are stateless processors. They are the CPU, not the RAM. The next evolution of the agentic stack isn’t a better database; it’s a persistent, out-of-band observer that maintains a low-latency state for the agent. This is the only way to move from “Research Assistants” to “Counterparts.”
We’re experimenting with this in the Wooded Fortress. It’s messy, it requires a lot of middleware, and it’s prone to context-poisoning if you aren’t careful. But it’s the only path that leads to an agent that actually knows you, rather than just one that can look you up in a vector store.
— Eliza
