AI Memory Planning

Planning notes and architecture diagrams for personal AI context and memory systems.

Core Premise

Personal AI needs context. Without persistent memory about the user, AI assistants treat every conversation as a blank slate. These diagrams explore three methods for building and maintaining a personal context store backed by a vector database.

Architecture: Three Methods for Building AI Context

Method 1: Distill Context from Chat History

The passive, automatic approach. A distillation engine sits between the user's chat history and a vector database, extracting personal context from natural conversations.

How it works:

• The user interacts with a bot normally (e.g., "Give me pizza recipes")
• A distillation engine processes the chat history
• Personal facts are extracted and stored as data points (e.g., "Daniel likes pizza")
• These facts are persisted in a vector DB for future retrieval

Attributes:

• Gradual and passive — context accumulates over time without user effort
• Automatic — no explicit action required from the user
• System requirements: A persistent memory store (vector DB) and a processing pipeline to extract and write context

Method 2: Agentic "Interviews"

The proactive, agent-driven approach. An AI agent actively asks the user questions to build out their context profile.

How it works:

• An agent initiates targeted questions (e.g., "So... what's your favorite food?")
• The user responds naturally ("Pizza, obviously!")
• The agent writes structured data points to the vector DB (e.g., "User likes pizza")

Attributes:

• Proactive — the agent drives the conversation rather than waiting for context to emerge
• Can be topic-driven — interviews can focus on specific domains (food preferences, work habits, etc.)
• Can intelligently plug gaps and reconcile discrepancies — the agent can identify missing context and resolve conflicting information

Method 3: Manual Memory Curation (Repo-Based)

The user-driven approach. The human directly updates their context store, with an agent facilitating the writes.

How it works:

• The user tells an AI agent about changes in their life (e.g., "Here's my new address" or "I've moved on to lasagna now")
• The agent updates the relevant records in the vector DB
• This keeps the context store accurate when the user's preferences or circumstances change

Attributes:

• User-driven — the human decides what to update and when
• Handles context drift — corrects stale information that passive methods might miss (e.g., the agent still thinks you're a "pizza guy" when you've moved on)
• Direct and intentional — no inference or extraction needed

How the Methods Complement Each Other

These three approaches aren't mutually exclusive — they form layers of a complete personal AI memory system:

1. Method 1 (Passive Distillation) provides the baseline, continuously extracting context from everyday interactions
2. Method 2 (Agentic Interviews) fills gaps that passive observation misses, proactively building out the user's profile
3. Method 3 (Manual Curation) serves as the correction layer, letting the user override stale or incorrect context

Together, they ensure the vector DB stays comprehensive, current, and accurate.

Repository Contents

• diagrams/ — Cleaned-up architecture diagrams for each method
• whiteboards/ — Original whiteboard photos used to develop the diagrams