How OpenMemory's Cognitive Architecture Works

While most AI memory solutions treat all data the same way, OpenMemory takes inspiration from cognitive science to create a more sophisticated memory system. Let's explore how our Hierarchical Memory Decomposition (HMD) architecture delivers superior performance and more human-like memory capabilities.

The Problem with Flat Vector Stores

Traditional vector databases store everything in a single dimensional space:

Query: "How do I debug JavaScript?"
Results: [Vector1, Vector2, Vector3, Vector4, Vector5]

This approach has several limitations:

• No semantic understanding of different types of information
• Equal treatment of facts vs. experiences vs. procedures
• Poor association between related but differently-typed memories
• No decay mechanism for irrelevant information
• Unexplainable results - why was this memory retrieved?

OpenMemory's Multi-Sector Approach

Instead of one flat space, OpenMemory organizes memories into cognitive sectors based on the type of information:

🧠 Semantic Memory

Facts, concepts, and general knowledge

{
  content: "JavaScript uses event-driven programming",
  sector: "semantic",
  embedding_semantic: [0.1, 0.8, -0.3, ...],
  salience: 0.9,
  created_at: "2025-01-01T10:00:00Z"
}

📖 Episodic Memory

Specific events and experiences

{
  content: "Yesterday I debugged a React component that wasn't re-rendering",
  sector: "episodic",
  embedding_episodic: [0.2, -0.1, 0.7, ...],
  salience: 0.7,
  created_at: "2025-01-05T14:30:00Z"
}

⚙️ Procedural Memory

Skills, processes, and how-to knowledge

{
  content: "To debug React: 1) Check props 2) Verify state updates 3) Use React DevTools",
  sector: "procedural",
  embedding_procedural: [-0.1, 0.4, 0.6, ...],
  salience: 0.8,
  created_at: "2025-01-03T09:15:00Z"
}

💝 Emotional Memory

Feelings, preferences, and emotional context

{
  content: "User finds debugging frustrating but rewarding when solved",
  sector: "emotional",
  embedding_emotional: [0.3, 0.2, -0.4, ...],
  salience: 0.6,
  created_at: "2025-01-05T14:45:00Z"
}

🤔 Reflective Memory

Meta-cognition and learned patterns

{
  content: "I notice I often miss obvious syntax errors when tired",
  sector: "reflective",
  embedding_reflective: [0.1, -0.2, 0.5, ...],
  salience: 0.5,
  created_at: "2025-01-06T11:20:00Z"
}

Hierarchical Memory Decomposition (HMD)

Our core architecture follows these principles:

1. One Canonical Node Per Memory

Each memory exists once in the system, preventing duplication and ensuring consistency.

2. Multiple Embeddings Per Memory

Depending on content, a memory can have embeddings in multiple sectors:

CREATE TABLE memories (
  id UUID PRIMARY KEY,
  content TEXT NOT NULL,
  user_id VARCHAR(255),
  salience REAL DEFAULT 0.5,
  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

CREATE TABLE memory_embeddings (
  memory_id UUID REFERENCES memories(id),
  sector VARCHAR(20) NOT NULL,
  embedding VECTOR(384) NOT NULL,
  PRIMARY KEY (memory_id, sector)
);

3. Single-Waypoint Linking

Memories connect through waypoint associations, creating a graph structure:

CREATE TABLE memory_associations (
  from_memory_id UUID REFERENCES memories(id),
  to_memory_id UUID REFERENCES memories(id),
  weight REAL DEFAULT 0.5,
  association_type VARCHAR(50),
  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (from_memory_id, to_memory_id)
);

4. Composite Similarity Scoring

Query results combine multiple factors:

function calculateMemoryScore(memory, query, userContext) {
  const semanticSim = cosineSimilarity(
    query.embedding,
    memory.embedding_semantic
  );
  const episodicSim = cosineSimilarity(
    query.embedding,
    memory.embedding_episodic
  );
  const proceduralSim = cosineSimilarity(
    query.embedding,
    memory.embedding_procedural
  );

  const maxSimilarity = Math.max(semanticSim, episodicSim, proceduralSim);
  const recencyScore = calculateRecency(memory.updated_at);
  const salienceScore = memory.salience;
  const associationBoost = calculateAssociationBoost(memory, userContext);

  return (
    0.6 * maxSimilarity +
    0.2 * salienceScore +
    0.1 * recencyScore +
    0.1 * associationBoost
  );
}

The Query Process

When a query comes in, OpenMemory follows this sophisticated retrieval process:

Step 1: Sectorization

Determine which memory sectors are relevant to the query:

async function sectorizeQuery(query) {
  const sectors = [];

  // Use ML classification or heuristics
  if (isFactualQuery(query)) sectors.push("semantic");
  if (isExperienceQuery(query)) sectors.push("episodic");
  if (isHowToQuery(query)) sectors.push("procedural");
  if (hasEmotionalContext(query)) sectors.push("emotional");
  if (isMetaCognitive(query)) sectors.push("reflective");

  return sectors;
}

Step 2: Multi-Sector Embedding

Generate embeddings for each relevant sector:

const queryEmbeddings = {};
for (const sector of relevantSectors) {
  queryEmbeddings[sector] = await generateEmbedding(query, { context: sector });
}

Step 3: Vector Search

Search within each sector's embedding space:

-- Search semantic memories
SELECT m.*, me.embedding,
       (me.embedding <=> $1) as similarity
FROM memories m
JOIN memory_embeddings me ON m.id = me.memory_id
WHERE me.sector = 'semantic'
  AND m.user_id = $2
ORDER BY similarity ASC
LIMIT 20;

Step 4: Waypoint Expansion

Expand results through one-hop associations:

async function expandThroughWaypoints(initialResults) {
  const expandedResults = [...initialResults];

  for (const result of initialResults) {
    const associations = await getAssociations(result.id);
    for (const assoc of associations) {
      const associatedMemory = await getMemory(assoc.to_memory_id);
      expandedResults.push({
        ...associatedMemory,
        associationWeight: assoc.weight,
        foundViaWaypoint: result.id,
      });
    }
  }

  return expandedResults;
}

Step 5: Composite Scoring

Rank all results using the composite scoring function and return the top-k matches.

Memory Decay and Reinforcement

OpenMemory implements natural memory decay that mimics human memory:

Exponential Decay

Salience decreases over time unless reinforced:

function updateSalience(memory, currentTime) {
  const timeSinceUpdate = currentTime - memory.updated_at;
  const decayRate = getDecayRate(memory.sector);

  const newSalience = memory.salience * Math.exp(-decayRate * timeSinceUpdate);
  return Math.max(newSalience, 0.1); // Minimum salience threshold
}

Reinforcement Through Access

Accessed memories get strengthened:

async function reinforceMemory(memoryId, reinforcementStrength = 0.1) {
  await db.query(
    `
    UPDATE memories 
    SET salience = LEAST(salience + $1, 1.0),
        updated_at = CURRENT_TIMESTAMP
    WHERE id = $2
  `,
    [reinforcementStrength, memoryId]
  );
}

Coactivation Patterns

Related memories get strengthened together:

async function reinforceCoactivatedMemories(accessedMemoryId) {
  const associations = await getStrongAssociations(accessedMemoryId);

  for (const assoc of associations) {
    const coactivationBoost = assoc.weight * 0.05;
    await reinforceMemory(assoc.to_memory_id, coactivationBoost);
  }
}

Association Building

OpenMemory automatically builds associations between related memories:

Similarity-Based Associations

async function buildSimilarityAssociations(newMemoryId) {
  const newMemory = await getMemory(newMemoryId);
  const candidates = await findSimilarMemories(newMemory, (threshold = 0.7));

  for (const candidate of candidates) {
    const weight = calculateAssociationWeight(newMemory, candidate);
    await createAssociation(newMemoryId, candidate.id, weight, "similarity");
  }
}

Temporal Associations

async function buildTemporalAssociations(newMemoryId) {
  const recentMemories = await getRecentMemoriesForUser(
    newMemory.user_id,
    (timeWindow = "1 hour")
  );

  for (const recent of recentMemories) {
    await createAssociation(newMemoryId, recent.id, 0.3, "temporal");
  }
}

User Pattern Associations

async function buildPatternAssociations(newMemoryId) {
  const userPatterns = await identifyUserPatterns(newMemory.user_id);

  for (const pattern of userPatterns) {
    if (matchesPattern(newMemory, pattern)) {
      await strengthenPatternAssociations(newMemoryId, pattern.memories);
    }
  }
}

Performance Optimizations

Several optimizations make this complex system performant:

Sector-Specific Indexes

-- Optimize vector searches per sector
CREATE INDEX idx_semantic_embeddings ON memory_embeddings
USING ivfflat (embedding vector_cosine_ops)
WHERE sector = 'semantic';

CREATE INDEX idx_episodic_embeddings ON memory_embeddings
USING ivfflat (embedding vector_cosine_ops)
WHERE sector = 'episodic';

Salience-Based Pruning

// Only search high-salience memories for fast queries
async function quickQuery(query, userContext) {
  return await searchMemories(query, {
    ...userContext,
    minSalience: 0.6,
    maxResults: 50,
  });
}

Background Processing

// Decay and association building happen asynchronously
setInterval(async () => {
  await processMemoryDecay();
  await buildNewAssociations();
  await cleanupLowSalienceMemories();
}, 60000); // Every minute

Why This Architecture Works

The cognitive architecture provides several key advantages:

🎯 Better Relevance

By understanding the type of information being queried, we can search in the most appropriate memory sectors.

🔗 Rich Context

Associations provide context that pure vector similarity misses, leading to more meaningful results.

⏰ Natural Forgetting

Memory decay prevents information overload and keeps the system focused on relevant, recent information.

📊 Explainable Results

We can trace exactly why a memory was retrieved - which sector, what associations, how the scoring worked.

🚀 Performance

Sector-specific searches are faster than searching one massive vector space.

The Future

We're continuously improving the architecture:

• Learned sectorization - ML models that automatically classify content types
• Dynamic sector weights - Adapting sector importance based on user behavior
• Hierarchical associations - Multi-level relationship modeling
• Attention mechanisms - Focus on the most relevant information
• Federated memory - Distributed memory across multiple nodes

Conclusion

OpenMemory's cognitive architecture represents a fundamental shift from flat vector storage to human-inspired memory systems. By understanding different types of information and how they relate, we achieve better relevance, performance, and explainability.

This architecture is why OpenMemory delivers 2-3× faster queries with 95% recall accuracy - it's not just storing vectors, it's building a cognitive model of memory itself.

---

Want to explore the technical implementation? Check out our GitHub repository or join the discussion on Discord.