The Problem With How AI Systems Remember
Modern large language models don’t really “remember” anything. They have a fixed context window — a kind of working notepad that gets erased between conversations. Some systems bolt on vector databases or retrieval pipelines as afterthoughts, but these feel exactly like what they are: external storage hacked onto a system that was never designed to use memory natively.
Biological brains don’t work this way. Your brain has at least three distinct memory systems that work in concert: a fleeting sensory buffer that holds the immediate present, an episodic system that stores sequences of lived experience, and a semantic network that crystallizes general knowledge over time. These aren’t separate databases — they’re deeply integrated, constantly exchanging information through a process neuroscientists call memory consolidation.
We just completed the first tier of FENA’s three-tier memory architecture, and it works nothing like what you’ve seen in conventional AI.
Three Tiers, Three Timescales
FENA’s memory system mirrors the biological hierarchy:
Working Memory: The FENA Active State Buffer. This is the system’s moment-to-moment awareness — a GPU-resident buffer that holds the current “active state” of the world as the system perceives it. But unlike a simple tensor cache, this buffer participates in FENA’s energy-based settling process. Information doesn’t just get written in — it has to “earn” its place by reducing the system’s overall free energy. Irrelevant information decays naturally as the system settles into coherent interpretations.
The active state buffer holds 512 slots, each a 512-dimensional vector. Think of it as a structured workspace where different aspects of the current situation compete for representation. A slot might hold the spatial layout of a scene, the emotional valence of a conversation, or a recently recalled fact — and the system’s settling dynamics determine which representations persist and which fade.
Episodic Memory: Experience as Trajectory. The second tier stores sequences of experience using FAISS (Facebook AI Similarity Search) on the CPU. But the key insight is what gets stored: not raw inputs, but compressed WorldState snapshots — the system’s interpretation of what happened, not just what it saw. This means retrieval is semantically meaningful. When the system encounters a new situation, it can find previous experiences that felt similar at a deep representational level, not just experiences that look superficially alike.
Semantic Memory: Crystallized Knowledge. The third tier uses frozen language model embeddings for long-term knowledge storage. This is the most stable layer — general facts and patterns that have been extracted from many episodes of experience. It’s CPU-resident, slow to update, but rich in the kind of background knowledge that makes intelligent behavior possible.
Why This Matters
The critical innovation isn’t any one tier — it’s how they interact. In conventional systems, memory retrieval is a separate “step” that happens before or after processing. In FENA, memory retrieval is part of the settling process itself. The system doesn’t first think and then remember — it thinks by remembering.
When FENA encounters a novel situation, the settling dynamics naturally pull in relevant episodic memories and semantic knowledge, because incorporating that information reduces free energy. The system finds useful memories not because it was told to search for them, but because remembering is energetically favorable.
This is memory that emerges from the physics of the system, not from architectural scaffolding. And it runs on consumer hardware — the entire three-tier system fits comfortably within 2GB of VRAM plus available system RAM.
What Comes Next
With the memory foundation in place, the next piece is the consolidation pipeline — the process by which fleeting working memory traces get consolidated into stable episodic memories, and repeated episodic patterns crystallize into semantic knowledge. This is where the system starts to genuinely learn from experience over time, building up knowledge in the way biological systems do: gradually, through repeated exposure, without being explicitly told what to remember.
The brain doesn’t have a “save” button. Neither will FENA.