MnemoCore

Infrastructure for Persistent Cognitive Memory

"Memory is not a container. It is a living process — a holographic continuum where every fragment contains the whole."

Quick Install

Option A — install from PyPI (recommended):

pip install mnemocore

Option B — install from source (for development):

git clone https://github.com/RobinALG87/MnemoCore-Infrastructure-for-Persistent-Cognitive-Memory.git
cd MnemoCore-Infrastructure-for-Persistent-Cognitive-Memory
python -m venv .venv && .\.venv\Scripts\activate   # Windows
# source .venv/bin/activate                        # Linux / macOS
pip install -e .          # editable install
pip install -e ".[dev]"   # + pytest, mypy, black, etc.

Set your API key before starting:
# Windows PowerShell
$env:HAIM_API_KEY = "your-secure-key"
# Linux / macOS
# export HAIM_API_KEY="your-secure-key"
Then start the API: uvicorn mnemocore.api.main:app --host 0.0.0.0 --port 8100

Full setup including Redis, Qdrant, Docker and configuration details are in Installation below.

What is MnemoCore?

MnemoCore is a research-grade cognitive memory infrastructure that gives AI agents a brain — not just a database.

Traditional vector stores retrieve. MnemoCore thinks. It is built on the mathematical framework of Binary Hyperdimensional Computing (HDC) and Vector Symbolic Architectures (VSA), principles rooted in Pentti Kanerva's landmark 2009 theory of cognitive computing. Every memory is encoded as a 16,384-dimensional binary holographic vector — a format that is simultaneously compact (2,048 bytes), noise-tolerant (Hamming geometry), and algebraically rich (XOR binding, majority bundling, circular permutation).

At its core lives the Holographic Active Inference Memory (HAIM) Engine — a system that does not merely answer queries, but:

Evaluates the epistemic novelty of every incoming memory before deciding to store it
Dreams — strengthening synaptic connections between related memories during idle cycles
Reasons by analogy — if king:man :: ?:woman, the VSA soul computes queen
Self-organizes into tiered storage based on biologically-inspired Long-Term Potentiation (LTP)
Scales from a single process to distributed nodes targeting 1B+ memories

Phase 4.x introduces cognitive enhancements including contextual masking, reliability feedback loops, semantic consolidation, gap detection/filling, temporal recall (episodic chaining + chrono-weighted query), a Subconscious Daemon with LLM-powered dream synthesis, and a full dependency-injection container pattern for clean modularity.

Architecture
Core Technology
The Memory Lifecycle
Tiered Storage
Phase 4.0 Cognitive Enhancements
Phase 4.4–4.5 Subconscious Daemon & LLM Integration
API Reference
Python Library Usage
Installation
Configuration
MCP Server Integration
Observability
Roadmap
Contributing

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        MnemoCore Stack                          │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│   ┌──────────────────────────────────────────────────────────┐  │
│   │              REST API  (FastAPI / Async)                  │  │
│   │   /store  /query  /feedback  /insights/gaps  /stats      │  │
│   │   Rate Limiting · API Key Auth · Prometheus Metrics      │  │
│   └─────────────────────────┬────────────────────────────────┘  │
│                             │                                   │
│   ┌─────────────────────────▼────────────────────────────────┐  │
│   │                  HAIM Engine                              │  │
│   │                                                          │  │
│   │   ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  │  │
│   │   │ Text Encoder │  │  EIG / Epist │  │  Subconsc.   │  │  │
│   │   │ (token→HDV)  │  │  Drive       │  │  Dream Loop  │  │  │
│   │   └──────────────┘  └──────────────┘  └──────────────┘  │  │
│   │                                                          │  │
│   │   ┌──────────────────────────────────────────────────┐  │  │
│   │   │            Binary HDV Core (VSA)                 │  │  │
│   │   │  XOR bind · majority_bundle · permute · Hamming  │  │  │
│   │   └──────────────────────────────────────────────────┘  │  │
│   └─────────────────────────┬────────────────────────────────┘  │
│                             │                                   │
│   ┌─────────────────────────▼────────────────────────────────┐  │
│   │                  Tier Manager                             │  │
│   │                                                          │  │
│   │   🔥 HOT         🌡 WARM            ❄️ COLD               │  │
│   │   In-Memory      Redis / mmap       Qdrant / Disk / S3   │  │
│   │   ≤2,000 nodes   ≤100,000 nodes     ∞ nodes              │  │
│   │   <1ms           <10ms              <100ms               │  │
│   └──────────────────────────────────────────────────────────┘  │
│                                                                 │
│   ┌─────────────────────────────────────────────────────────┐   │
│   │                  Conceptual Layer ("The Soul")           │   │
│   │   ConceptualMemory · Analogy Engine · Symbol Algebra     │   │
│   └─────────────────────────────────────────────────────────┘   │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

Component Overview

Component	File	Responsibility
HAIM Engine	`src/mnemocore/core/engine.py`	Central cognitive coordinator — store, query, dream, delete
BinaryHDV	`src/mnemocore/core/binary_hdv.py`	16384-dim binary vector math (XOR, Hamming, bundle, permute)
TextEncoder	`src/mnemocore/core/binary_hdv.py`	Token→HDV pipeline with positional permutation binding
MemoryNode	`src/mnemocore/core/node.py`	Memory unit with LTP, epistemic values, tier state
TierManager	`src/mnemocore/core/tier_manager.py`	HOT/WARM/COLD orchestration with LTP-driven eviction
SynapticConnection	`src/mnemocore/core/synapse.py`	Hebbian synapse with strength, decay, and fire tracking
SynapseIndex	`src/mnemocore/core/synapse_index.py`	Fast synapse lookup index for associative spreading
ConceptualMemory	`src/mnemocore/core/holographic.py`	VSA soul for analogy and cross-domain symbolic reasoning
AsyncRedisStorage	`src/mnemocore/core/async_storage.py`	Async Redis backend (WARM tier + pub/sub)
BayesianLTP	`src/mnemocore/core/bayesian_ltp.py`	Bayesian reliability scoring on top of LTP strength
SemanticConsolidation	`src/mnemocore/core/semantic_consolidation.py`	Memory deduplication via majority-bundle prototyping
ConsolidationWorker	`src/mnemocore/core/consolidation_worker.py`	Async worker scheduling nightly consolidation
GapDetector	`src/mnemocore/core/gap_detector.py`	Temporal co-occurrence analysis for knowledge gaps
GapFiller	`src/mnemocore/core/gap_filler.py`	Bridge detected gaps via synapse creation
Immunology	`src/mnemocore/core/immunology.py`	Auto-associative attractor cleanup for vector drift
Attention	`src/mnemocore/core/attention.py`	XOR context masking / project isolation
BatchOps	`src/mnemocore/core/batch_ops.py`	Vectorized bulk store / query operations
HNSWIndex	`src/mnemocore/core/hnsw_index.py`	In-process HNSW approximate nearest-neighbour index
QdrantStore	`src/mnemocore/core/qdrant_store.py`	Async Qdrant COLD tier backend
RecursiveSynthesizer	`src/mnemocore/core/recursive_synthesizer.py`	Deep concept synthesis via iterative VSA composition
RippleContext	`src/mnemocore/core/ripple_context.py`	Cascading context propagation across synaptic graph
SubconsciousAI	`src/mnemocore/core/subconscious_ai.py`	LLM-guided dream synthesis worker
SubconsciousDaemon	`src/mnemocore/subconscious/daemon.py`	Background process orchestrating dream/consolidation cycles
LLMIntegration	`src/mnemocore/llm_integration.py`	Agent-facing LLM connector (OpenAI / Anthropic compatible)
Container	`src/mnemocore/core/container.py`	Dependency-injection wiring for all core components
GoalTree	`src/mnemocore/meta/goal_tree.py`	Hierarchical goal / task tracking for meta-cognition
LearningJournal	`src/mnemocore/meta/learning_journal.py`	Persistent log of what the agent has learned over time
API	`src/mnemocore/api/main.py`	FastAPI REST interface with async wrappers and middleware
MCP Server	`src/mnemocore/mcp/server.py`	Model Context Protocol adapter for agent tool integration

Core Technology: Binary HDV & VSA

MnemoCore's mathematical foundation is Hyperdimensional Computing — a computing paradigm that encodes information in very high-dimensional binary vectors (HDVs), enabling noise-tolerant, distributed, and algebraically composable representations.

The Vector Space

Every piece of information — a word, a sentence, a concept, a goal — is encoded as a 16,384-dimensional binary vector:

Dimension D = 16,384 bits = 2,048 bytes per vector
Storage:      packed as numpy uint8 arrays
Similarity:   Hamming distance (popcount of XOR result)
Random pair:  ~50% similarity (orthogonality by probability)

At this dimensionality, two random vectors will differ in ~50% of bits. This near-orthogonality is the foundation of the system's expressive power — related concepts cluster together while unrelated ones remain maximally distant.

VSA Algebra

Four primitive operations make the entire system work:

Binding — XOR `⊕`

Creates an association between two concepts. Crucially, the result is dissimilar to both inputs (appears as noise), making it a true compositional operation.

# Bind content to its context
bound = content_vec.xor_bind(context_vec)  # content ⊕ context

# Self-inverse: unbind by re-binding
recovered = bound.xor_bind(context_vec)   # ≈ content (XOR cancels)

Key mathematical properties:

Self-inverse: A ⊕ A = 0 (XOR cancels itself)
Commutative: A ⊕ B = B ⊕ A
Distance-preserving: hamming(A⊕C, B⊕C) = hamming(A, B)

Bundling — Majority Vote

Creates a prototype that is similar to all inputs. This is how multiple memories combine into a concept.

from mnemocore.core.binary_hdv import majority_bundle

# Create semantic prototype from related memories
concept = majority_bundle([vec_a, vec_b, vec_c, vec_d])  # similar to all inputs

Permutation — Circular Shift

Encodes sequence and roles without separate positional embeddings.

# Positional encoding: token at position i
positioned = token_vec.permute(shift=i)  # circular bit-shift

# Encode "hello world" with order information
hello_positioned = encoder.get_token_vector("hello").permute(0)
world_positioned = encoder.get_token_vector("world").permute(1)
sentence_vec = majority_bundle([hello_positioned, world_positioned])

Similarity — Hamming Distance

Fast comparison using vectorized popcount over XOR results:

# Normalized similarity: 1.0 = identical, 0.5 = unrelated
sim = vec_a.similarity(vec_b)  # 1.0 - hamming(a, b) / D

# Batch nearest-neighbor search (no Python loops)
distances = batch_hamming_distance(query, database_matrix)

Text Encoding Pipeline

The TextEncoder converts natural language to HDVs using a token-position binding scheme:

"Python TypeError" →
  token_hdv("python") ⊕ permute(0)  =  positioned_0
  token_hdv("typeerror") ⊕ permute(1)  =  positioned_1
  majority_bundle([positioned_0, positioned_1])  =  final_hdv

Token vectors are deterministic — seeded via SHAKE-256 hash — meaning the same word always produces the same base vector, enabling cross-session consistency without a vocabulary file.

The Memory Lifecycle

Every memory passes through a defined lifecycle from ingestion to long-term storage:

Incoming Content
      │
      ▼
 ┌─────────────┐
 │ TextEncoder │  → 16,384-dim binary HDV
 └──────┬──────┘
        │
        ▼
 ┌──────────────────┐
 │ Context Binding  │  → XOR bind with goal_context if present
 │  (XOR)           │     bound_vec = content ⊕ context
 └──────┬───────────┘
        │
        ▼
 ┌──────────────────┐
 │  EIG Evaluation  │  → Epistemic Information Gain
 │  (Novelty Check) │     eig = normalized_distance(vec, context_vec)
 └──────┬───────────┘     tag "epistemic_high" if eig > threshold
        │
        ▼
 ┌─────────────────┐
 │  MemoryNode     │  → id, hdv, content, metadata
 │  Creation       │     ltp_strength = I × log(1+A) × e^(-λT)
 └──────┬──────────┘
        │
        ▼
 ┌─────────────────┐
 │  HOT Tier       │  → In-memory dict (max 2000 nodes)
 │  (RAM)          │     LTP eviction: low-LTP nodes → WARM
 └──────┬──────────┘
        │     (background)
        ▼
 ┌─────────────────┐
 │ Subconscious    │  → Dream cycle fires
 │ Dream Loop      │     Query similar memories
 └──────┬──────────┘     Strengthen synapses (Hebbian)
        │
        ▼
 ┌─────────────────┐
 │  WARM Tier      │  → Redis-backed persistence
 │  (Redis/mmap)   │     async dual-write + pub/sub events
 └──────┬──────────┘
        │     (scheduled, nightly)
        ▼
 ┌─────────────────┐
 │  COLD Tier      │  → Qdrant / Disk / S3
 │  (Archival)     │     ANN search, long-term persistence
 └─────────────────┘

Long-Term Potentiation (LTP)

Memories are not equal. Importance is computed dynamically using a biologically-inspired LTP formula:

S = I × log(1 + A) × e^(-λ × T)

Where:
  S = LTP strength (determines tier placement)
  I = Importance (derived from epistemic + pragmatic value)
  A = Access count (frequency of retrieval)
  λ = Decay lambda (configurable, default ~0.01)
  T = Age in days

Memories with high LTP remain in HOT tier. Those that decay are automatically promoted to WARM, then COLD — mirroring how biological memory consolidates from working memory to long-term storage.

Synaptic Connections

Memories are linked by SynapticConnection objects that implement Hebbian learning: "neurons that fire together, wire together."

Every time two memories are co-retrieved (via the background dream loop or explicit binding), their synaptic strength increases. During query time, synaptic spreading amplifies scores of connected memories even when they do not directly match the query vector — enabling associative recall.

# Explicit synapse creation
engine.bind_memories(id_a, id_b, success=True)

# Associative spreading: query top seeds spread activation to neighbors
# neighbor_score += seed_score × synapse_strength × 0.3

Tiered Storage: HOT / WARM / COLD

Tier	Backend	Capacity	Latency	Eviction Trigger
🔥 HOT	Python dict (RAM)	2,000 nodes	< 1ms	LTP < threshold
🌡 WARM	Redis + mmap	100,000 nodes	< 10ms	Age + low access
❄️ COLD	Qdrant / Disk / S3	Unlimited	< 100ms	Manual / scheduled

Promotion is automatic: accessing a WARM or COLD memory re-promotes it to HOT based on recalculated LTP. Eviction is LRU-weighted by LTP strength — the most biologically active memories always stay hot.

Phase 4.0 Cognitive Enhancements

MnemoCore Phase 4.0 introduces five architectural enhancements that elevate the system from data retrieval to cognitive reasoning. Full implementation specifications are in COGNITIVE_ENHANCEMENTS.md.

1. Contextual Query Masking (XOR Attention)

Problem: Large multi-project deployments suffer from cross-context interference. A query for "Python error handling" returns memories from all projects equally, diluting precision.

Solution: Bidirectional XOR context binding — apply the same context vector at both storage and query time:

Store:  bound_vec   = content ⊕ context_vec
Query:  masked_query = query   ⊕ context_vec

Result: (content ⊕ C) · (query ⊕ C) ≈ content · query
        (context cancels, cross-project noise is suppressed)

# Store memories in a project context
engine.store("API rate limiting logic", goal_id="ProjectAlpha")
engine.store("Garden watering schedule", goal_id="HomeProject")

# Query with context mask — only ProjectAlpha memories surface
results = engine.query("API logic", top_k=5, context="ProjectAlpha")

Expected impact: +50–80% query precision (P@5) in multi-project deployments.

2. Reliability Feedback Loop (Self-Correcting Memory)

Problem: Wrong or outdated memories persist with the same retrieval weight as correct ones. The system has no mechanism to learn from its own mistakes.

Solution: Bayesian reliability scoring with real-world outcome feedback:

reliability = (successes + 1) / (successes + failures + 2)  # Laplace smoothing

LTP_enhanced = I × log(1+A) × e^(-λT) × reliability

# After using a retrieved memory:
engine.provide_feedback(memory_id, outcome=True)   # Worked → boost reliability
engine.provide_feedback(memory_id, outcome=False)  # Failed → reduce reliability

# System auto-tags consistently wrong memories as "unreliable"
# and verified memories (>5 successes, >0.8 score) as "verified"

The system converges toward high-confidence knowledge — memories that have demonstrably worked in practice rank above theoretically similar but unproven ones.

3. Semantic Memory Consolidation (Dream-Phase Synthesis)

Problem: Episodic memory grows without bound. 1,000 memories about "Python TypeError" are semantically equivalent but consume 2MB of vector space and slow down linear scan queries.

Solution: Nightly ConsolidationWorker clusters similar WARM tier memories and replaces them with a semantic anchor — a majority-bundled prototype:

BEFORE consolidation:
  mem_001: "Python TypeError in line 45"    (2KB vector)
  mem_002: "TypeError calling function"     (2KB vector)
  ...   ×100 similar memories              (200KB total)

AFTER consolidation:
  anchor_001: "Semantic pattern: python typeerror function"
              metadata: {source_count: 100, confidence: 0.94}
              hdv: majority_bundle([mem_001.hdv, ..., mem_100.hdv])  (2KB)

# Manual trigger (runs automatically at 3 AM)
stats = engine.trigger_consolidation()
# → {"abstractions_created": 12, "memories_consolidated": 847}

# Via API (admin endpoint)
POST /admin/consolidate

Expected impact: 70–90% memory footprint reduction, 10x query speedup at scale.

4. Auto-Associative Cleanup Loop (Vector Immunology)

Problem: Holographic vectors degrade over time through repeated XOR operations, noise accumulation, and long-term storage drift. After months of operation, retrieved vectors become "blurry" and similarity scores fall.

Solution: Iterative attractor dynamics — when a retrieved vector appears noisy, snap it to the nearest stable concept in a codebook of high-confidence prototypes:

noisy_vec → find K nearest in codebook
         → majority_bundle(K neighbors)
         → check convergence (Hamming distance < 5%)
         → iterate until converged or max iterations reached

# Cleanup runs automatically on retrieval when noise > 15%
node = engine.get_memory(memory_id, auto_cleanup=True)
# node.metadata["cleaned"] = True  (if cleanup was triggered)
# node.metadata["cleanup_iterations"] = 3

# Codebook is auto-populated from most-accessed, high-reliability memories

Expected impact: Maintain >95% similarity fidelity even after years of operation.

5. Knowledge Gap Detection (Proactive Curiosity)

Problem: The system is entirely reactive — it answers queries but never identifies what it doesn't know. True cognitive autonomy requires self-directed learning.

Solution: Temporal co-occurrence analysis — detect concepts that are frequently accessed close in time but have no synaptic connection, flagging them as knowledge gaps:

# Automatically runs hourly
gaps = engine.detect_knowledge_gaps(time_window_seconds=300)

# Returns structured insight:
# [
#   {
#     "concept_a": "Python asyncio event loop",
#     "concept_b": "FastAPI dependency injection",
#     "suggested_query": "How does asyncio relate to FastAPI dependency injection?",
#     "co_occurrence_count": 4
#   }
# ]

# Query endpoint
GET /insights/gaps?lookback_hours=24

# Fill gap manually (or via LLM agent)
POST /insights/fill-gap
{"concept_a_id": "mem_xxx", "concept_b_id": "mem_yyy",
 "explanation": "FastAPI uses asyncio's event loop internally..."}

The system becomes capable of saying what it doesn't understand and requesting clarification — the first step toward genuine cognitive autonomy.

Phase 4.4–4.5: Subconscious Daemon & LLM Integration

Subconscious Daemon (Autonomous Background Mind)

Phase 4.4 introduced SubconsciousAI — a worker that fires during idle cycles and calls an external LLM to generate synthetic dream memories: structured insights derived by reasoning over existing memory clusters, rather than through direct observation.

Phase 4.5 hardened this into a full SubconsciousDaemon — an independently managed asyncio process that orchestrates dream cycles, consolidation scheduling, and subconscious queue processing:

# The daemon is started automatically when the API starts up.
# It coordinates:
#   - Dream synthesis: SubconsciousAI → LLM → synthetic insights stored back
#   - Consolidation scheduling: ConsolidationWorker fired on a configurable interval
#   - Subconscious queue: novelty detection from Redis pub/sub stream

Configure in config.yaml:

haim:
  subconscious_ai:
    enabled: true
    api_url: "https://api.openai.com/v1/chat/completions"  # or Anthropic
    model: "gpt-4o-mini"
    # api_key: set via SUBCONSCIOUS_AI_API_KEY env var
    dream_interval_seconds: 300
    batch_size: 5

Dependency Injection Container

All major services (TierManager, AsyncRedisStorage, QdrantStore, SubconsciousAI, etc.) are now wired through src/mnemocore/core/container.py. This eliminates global singleton state and makes every subsystem testable in isolation:

from mnemocore.core.container import build_container

container = build_container(config)
engine   = container.engine()
tier_mgr = container.tier_manager()

LLM Agent Integration

src/mnemocore/llm_integration.py provides a high-level interface for attaching MnemoCore to any OpenAI/Anthropic-style LLM agent loop:

from mnemocore.llm_integration import MnemoCoreAgent

agent = MnemoCoreAgent(engine)

# Store agent observations
agent.observe("User prefers concise answers over verbose ones")

# Recall relevant context before a response
context = agent.recall("user preference", top_k=3)

API Reference

Authentication

All endpoints require an API key via the X-API-Key header:

export HAIM_API_KEY="your-secure-key"
curl -H "X-API-Key: $HAIM_API_KEY" ...

Endpoints

`POST /store`

Store a new memory with optional context binding.

Request:
{
  "content": "FastAPI uses Pydantic v2 for request validation.",
  "metadata": {"source": "docs", "tags": ["python", "fastapi"]},
  "context": "ProjectAlpha",
  "agent_id": "agent-001",
  "ttl": 3600
}

Response:
{
  "ok": true,
  "memory_id": "mem_1739821234567",
  "message": "Stored memory: mem_1739821234567"
}

`POST /query`

Query memories by semantic similarity with optional context masking.

Request:
{
  "query": "How does FastAPI handle request validation?",
  "top_k": 5,
  "context": "ProjectAlpha"
}

Response:
{
  "ok": true,
  "query": "How does FastAPI handle request validation?",
  "results": [
    {
      "id": "mem_1739821234567",
      "content": "FastAPI uses Pydantic v2 for request validation.",
      "score": 0.8923,
      "metadata": {"source": "docs"},
      "tier": "hot"
    }
  ]
}

`POST /feedback`

Report outcome of a retrieved memory (Phase 4.0 reliability loop).

Request:
{
  "memory_id": "mem_1739821234567",
  "outcome": true,
  "comment": "This solution worked perfectly."
}

Response:
{
  "ok": true,
  "memory_id": "mem_1739821234567",
  "reliability_score": 0.714,
  "success_count": 4,
  "failure_count": 1
}

`GET /memory/{memory_id}`

Retrieve a specific memory with full metadata.

Response:
{
  "id": "mem_1739821234567",
  "content": "...",
  "metadata": {...},
  "created_at": "2026-02-17T20:00:00Z",
  "ltp_strength": 1.847,
  "epistemic_value": 0.73,
  "reliability_score": 0.714,
  "tier": "hot"
}

`DELETE /memory/{memory_id}`

Delete memory from all tiers and clean up synapses.

`POST /concept`

Define a symbolic concept for analogical reasoning.

{"name": "king", "attributes": {"gender": "man", "role": "ruler", "domain": "royalty"}}

`POST /analogy`

Solve analogies using VSA algebra: source:value :: target:?

Request:  {"source_concept": "king", "source_value": "man", "target_concept": "queen"}
Response: {"results": [{"value": "woman", "score": 0.934}]}

`GET /insights/gaps`

Detect knowledge gaps from recent temporal co-activity (Phase 4.0).

Response:
{
  "gaps_detected": 3,
  "knowledge_gaps": [
    {
      "concept_a": "asyncio event loop",
      "concept_b": "FastAPI middleware",
      "suggested_query": "How does event loop relate to middleware?",
      "co_occurrence_count": 5
    }
  ]
}

`POST /admin/consolidate`

Trigger manual semantic consolidation (normally runs automatically at 3 AM).

`GET /stats`

Engine statistics — tiers, synapse count, consolidation state.

`GET /health`

Health check — Redis connectivity, engine readiness, degraded mode status.

`GET /metrics`

Prometheus metrics endpoint.

Python Library Usage

Basic Store and Query

from mnemocore.core.engine import HAIMEngine

engine = HAIMEngine(persist_path="./data/memory.jsonl")

# Store memories
engine.store("Python generators are lazy iterators", metadata={"topic": "python"})
engine.store("Use 'yield' to create generator functions", metadata={"topic": "python"})
engine.store("Redis XADD appends to a stream", goal_id="infrastructure")

# Query (global)
results = engine.query("How do Python generators work?", top_k=3)
for mem_id, score in results:
    mem = engine.get_memory(mem_id)
    print(f"[{score:.3f}] {mem.content}")

# Query with context masking
results = engine.query("data streams", top_k=5, context="infrastructure")

engine.close()

Analogical Reasoning

# Define concepts
engine.define_concept("king",  {"gender": "man",   "role": "ruler"})
engine.define_concept("queen", {"gender": "woman", "role": "ruler"})
engine.define_concept("man",   {"gender": "man"})

# VSA analogy: king:man :: ?:woman → queen
result = engine.reason_by_analogy(
    src="king", val="man", tgt="woman"
)
print(result)  # [("queen", 0.934), ...]

Working with the Binary HDV Layer Directly

from mnemocore.core.binary_hdv import BinaryHDV, TextEncoder, majority_bundle

encoder = TextEncoder(dimension=16384)

# Encode text
python_vec  = encoder.encode("Python programming")
fastapi_vec = encoder.encode("FastAPI framework")
error_vec   = encoder.encode("error handling")

# Bind concept to role
python_in_fastapi = python_vec.xor_bind(fastapi_vec)

# Bundle multiple concepts into prototype
web_dev_prototype = majority_bundle([python_vec, fastapi_vec, error_vec])

# Similarity
print(python_vec.similarity(web_dev_prototype))  # High (part of bundle)
print(python_vec.similarity(error_vec))          # ~0.5 (unrelated)

# Batch nearest-neighbor search
from mnemocore.core.binary_hdv import batch_hamming_distance
import numpy as np

database = np.stack([v.data for v in [python_vec, fastapi_vec, error_vec]])
distances = batch_hamming_distance(python_vec, database)

Reliability Feedback Loop

mem_id = engine.store("Always use asyncio.Lock() in async code, not threading.Lock()")
results = engine.query("async locking")

# It works — report success
engine.provide_feedback(mem_id, outcome=True, comment="Solved deadlock issue")

# Over time, high-reliability memories get 'verified' tag
# and are ranked above unproven ones in future queries

Semantic Consolidation

stats = engine.trigger_consolidation()
print(f"Created {stats['abstractions_created']} semantic anchors")
print(f"Consolidated {stats['memories_consolidated']} episodic memories")

# Automatic: runs every night at 3 AM via background asyncio task

Installation

Prerequisites

Python 3.10+
Redis 6+ — Required for WARM tier and async event streaming
Qdrant (optional) — For COLD tier at billion-scale
Docker (recommended) — For Redis and Qdrant services

Quick Start

# 1. Clone
git clone https://github.com/RobinALG87/MnemoCore-Infrastructure-for-Persistent-Cognitive-Memory.git
cd MnemoCore-Infrastructure-for-Persistent-Cognitive-Memory

# 2. Create virtual environment
python -m venv .venv
.\.venv\Scripts\activate          # Windows (PowerShell)
# source .venv/bin/activate       # Linux / macOS

# 3. Install (recommended — uses pyproject.toml as canonical source)
pip install -e .

# Or install runtime deps only (Docker / legacy):
# pip install -r requirements.txt

# To include dev tools (pytest, mypy, black, etc.):
pip install -e ".[dev]"

# 4. Start Redis
docker run -d -p 6379:6379 redis:7.2-alpine

# 5. Set API key (never hardcode — use env var or .env file)
# Windows PowerShell:
$env:HAIM_API_KEY = "your-secure-key-here"
# Linux / macOS:
# export HAIM_API_KEY="your-secure-key-here"

# 6. Start the API
uvicorn mnemocore.api.main:app --host 0.0.0.0 --port 8100

The API is now live at http://localhost:8100. Visit http://localhost:8100/docs for the interactive Swagger UI.

Using the .env file

Copy the provided template and fill in your values — the API and docker-compose both pick it up automatically:

cp .env.example .env
# Edit .env and set HAIM_API_KEY, REDIS_URL, etc.

Note: .env is listed in .gitignore and must never be committed. Only .env.example (with placeholder values) belongs in version control.

Full Stack with Docker Compose

# Requires .env with HAIM_API_KEY set
docker compose up -d

This starts MnemoCore, Redis 7.2, and Qdrant in one command.

With Qdrant (Phase 4.x Scale)

# Start Qdrant alongside Redis
docker run -d -p 6333:6333 qdrant/qdrant

# Enable in config.yaml
qdrant:
  enabled: true
  host: localhost
  port: 6333

Configuration

All configuration lives in config.yaml. Sensitive values can be overridden with environment variables — the config loader looks for HAIM_-prefixed vars and also honours per-service overrides like HAIM_API_KEY, REDIS_PASSWORD, QDRANT_API_KEY, HAIM_CORS_ORIGINS, and SUBCONSCIOUS_AI_API_KEY.

haim:
  version: "4.5"
  dimensionality: 16384        # Binary vector dimensions (must be multiple of 64)

  encoding:
    mode: "binary"             # "binary" (recommended) or "float" (legacy, deprecated)
    token_method: "bundle"     # "bundle" (XOR+permute) or "hash"

  tiers:
    hot:
      max_memories: 2000       # Max nodes in RAM
      ltp_threshold_min: 0.7   # Evict below this LTP strength
      eviction_policy: "lru"
    warm:
      max_memories: 100000     # Max nodes in Redis/mmap
      ltp_threshold_min: 0.3
    cold:
      storage_backend: "filesystem"   # "filesystem" or "s3"
      compression: "gzip"

  ltp:
    initial_importance: 0.5
    decay_lambda: 0.01         # Higher = faster forgetting
    permanence_threshold: 0.95 # LTP above this is immune to decay
    half_life_days: 30.0

  hysteresis:
    promote_delta: 0.15        # LTP must exceed threshold by this much to promote
    demote_delta: 0.10

  redis:
    url: "redis://localhost:6379/0"
    stream_key: "haim:subconscious"
    max_connections: 10
    socket_timeout: 5
    # password: set via REDIS_PASSWORD env var

  qdrant:
    url: "http://localhost:6333"
    collection_hot:  "haim_hot"
    collection_warm: "haim_warm"
    enabled: false
    # api_key: set via QDRANT_API_KEY env var

  security:
    # api_key: set via HAIM_API_KEY env var — never hardcode here
    cors_origins: ["http://localhost:3000"]

  subconscious_ai:
    enabled: false
    api_url: "https://api.openai.com/v1/chat/completions"
    model: "gpt-4o-mini"
    dream_interval_seconds: 300
    batch_size: 5
    # api_key: set via SUBCONSCIOUS_AI_API_KEY env var

  observability:
    metrics_port: 9090
    log_level: "INFO"
    structured_logging: true

  paths:
    data_dir: "./data"
    memory_file: "./data/memory.jsonl"
    codebook_file: "./data/codebook.json"
    concepts_file: "./data/concepts.json"
    synapses_file: "./data/synapses.json"
    warm_mmap_dir: "./data/warm_tier"
    cold_archive_dir: "./data/cold_archive"

  mcp:
    enabled: false
    transport: "stdio"
    host: "127.0.0.1"
    port: 8110
    api_base_url: "http://localhost:8100"

Security Note

MnemoCore requires an explicit API key. There is no default fallback key in production builds.

# Generate a cryptographically secure key:
python -c "import secrets; print(secrets.token_urlsafe(32))"

# Set it (never commit this value):
export HAIM_API_KEY="<generated-value>"

MCP Server Integration

MnemoCore exposes a Model Context Protocol (MCP) server, enabling direct integration with Claude, GPT-4, and any MCP-compatible agent framework.

Setup

# Start API first
uvicorn mnemocore.api.main:app --host 0.0.0.0 --port 8100

# Configure MCP in config.yaml
haim:
  mcp:
    enabled: true
    transport: "stdio"  # or "sse" for streaming

# Run MCP server
python -m mnemocore.mcp.server

Claude Desktop Configuration

Add to your Claude Desktop config.json:

{
  "mcpServers": {
    "mnemocore": {
      "command": "python",
      "args": ["-m", "mnemocore.mcp.server"],
      "env": {
        "HAIM_API_KEY": "your-key",
        "HAIM_BASE_URL": "http://localhost:8100"
      }
    }
  }
}

Once connected, the agent can:

store_memory(content, context) — persist learned information
query_memory(query, context, top_k) — recall relevant memories
provide_feedback(memory_id, outcome) — signal what worked
get_knowledge_gaps() — surface what it doesn't understand

Observability

MnemoCore ships with built-in Prometheus metrics and structured logging.

Prometheus Metrics

Available at GET /metrics:

Metric	Description
`haim_api_request_count`	Total requests by endpoint and status
`haim_api_request_latency_seconds`	Request latency histogram
`haim_storage_operation_count`	Store/query/delete operations
`haim_hot_tier_size`	Current HOT tier memory count
`haim_synapse_count`	Active synaptic connections

Grafana Dashboard

A sample Grafana dashboard config is available at grafana-dashboard.json in the repository root. Import it directly into Grafana via Dashboards → Import → Upload JSON file.

Structured Logging

All components use structured Python logging with contextual fields:

2026-02-17 20:00:00 INFO  Stored memory mem_1739821234567 (EIG: 0.7823)
2026-02-17 20:00:01 INFO  Memory mem_1739821234567 reliability updated: 0.714 (4✓ / 1✗)
2026-02-17 03:00:00 INFO  Consolidation complete: abstractions_created=12, consolidated=847
2026-02-17 04:00:00 INFO  Knowledge gap detected: asyncio ↔ FastAPI middleware (5 co-occurrences)

Testing

# Run full test suite
pytest

# Run with coverage
pytest --cov=src --cov-report=html

# Run specific feature tests
pytest tests/test_xor_attention.py         # Contextual masking
pytest tests/test_stability.py             # Reliability/Bayesian stability
pytest tests/test_consolidation.py         # Semantic consolidation
pytest tests/test_engine_cleanup.py        # Cleanup and decay
pytest tests/test_phase43_regressions.py   # Phase 4.3 regression guardrails
pytest tests/test_tier_manager.py          # Tier demotion / promotion logic
pytest tests/test_dream_loop.py            # Subconscious dream loop
pytest tests/test_subconscious_ai_worker.py # LLM-powered dream worker (if offline: uses mocks)
pytest tests/test_recursive_synthesizer.py  # Deep concept synthesis
pytest tests/test_batch_ops.py             # Bulk ingestion operations
pytest tests/test_mcp_server.py            # MCP server adapter

# End-to-end flow
pytest tests/test_e2e_flow.py -v

Roadmap

Current Release (v4.5.0)

Binary HDV core (XOR bind / bundle / permute / Hamming)
Three-tier HOT/WARM/COLD memory lifecycle
Async API + MCP integration
XOR attention masking + Bayesian reliability updates
Semantic consolidation, immunology cleanup, and gap detection/filling
Temporal recall: episodic chaining + chrono-weighted query
Regression guardrails for Phase 4.3 critical paths
Phase 4.4 — Subconscious AI Worker (LLM-powered dream synthesis)
Phase 4.5 — Subconscious Daemon, persistence hardening, tier-manager demotion race fix
Dependency-injection Container pattern (replaces singleton)
HNSW in-process index for hot-tier ANN search
Batch operations for bulk ingestion
Meta-cognition layer: GoalTree + LearningJournal

Next Steps

Hardening pass for distributed/clustered HOT-tier behavior
Extended observability standardization (mnemocore_* metric prefix across all components)
Self-improvement loop (design documented in docs/SELF_IMPROVEMENT_DEEP_DIVE.md, staged rollout pending)
CUDA kernels for batch HDV operations at scale
Helm chart production hardening (resource autoscaling, PodDisruptionBudget)

Contributing

MnemoCore is an active research project. Contributions are welcome — especially:

Performance: CUDA kernels, FAISS integration, async refactoring
Algorithms: Better clustering for consolidation, improved EIG formulas
Integrations: New storage backends, LLM connectors
Tests: Coverage for edge cases, property-based testing

Process

# Fork and clone
git checkout -b feature/your-feature-name

# Make changes, ensure tests pass
pytest

# Commit with semantic message
git commit -m "feat(consolidation): add LLM-powered prototype labeling"

# Open PR — describe the what, why, and performance impact

Please follow the implementation patterns established in docs/ARCHITECTURE.md and docs/ROADMAP.md for architectural guidance, and review CHANGELOG.md to understand what has already landed.

License

MIT License — see LICENSE for details.

Contact

Robin Granberg
📧 robin@veristatesystems.com

Building the cognitive substrate for the next generation of autonomous AI.

Downloads last month: 4

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support