dsh-memory Architecture

Version: 0.2.0 · License: MIT · Changelog

Overview

Cross-session memory plugin for DeepSeek Harness (dsh):

• Persist preferences, lessons, project facts across sessions
• Retrieve by keyword and semantic (embedding) search
• Inject automatically so the agent "naturally carries" memory
• Stay safe: no data loss on concurrent writes or corrupt files

Module Layout

src/
├── index.ts    # plugin entry: tools + session-first injection + pre-step listener
├── storage.ts  # Storage interface + YamlStorage (write queue, mtime cache, quarantine)
├── search.ts   # KeywordSearch / EmbeddingSearch / HybridSearch (RRF) / OllamaEmbedder
├── inject.ts   # selectForInjection / renderSection (title-first) / deriveTitle / entryTitle
├── dynamic.ts  # DynamicInjector (per-session dedup + retrieval gate + fixed-set exclusion)
└── types.ts    # data model (+ reserved fields: vector/scope/weight)

Layering: Storage and SearchEngine are interfaces — YAML is the v1 backend, keyword + embedding the v1/v2 search; both can be swapped without touching the rest.

Data Model

- id: MEM-YYYYMMDD-NNN
  title: "Summary title (auto-derived if absent)"   # injected as title
  content: "..."
  category: preference | project | lesson | fact
  tags: [tag1, tag2]
  importance: high | normal | low
  created: YYYY-MM-DD
  updated: YYYY-MM-DD
  source: user | agent | conversation
  retired: false
  forceInject: false           # full behavioural rules, always resident
  injectLevel: session | auto  # resident vs on-demand
  vector: [...]   # embedding, auto-computed on write

Reserved fields: vector (active), scope (multi-project), weight (feedback scoring).

Injection

Two channels:

1. Session-level (injectLevel: session + forceInject): injected ONCE on the session's first agent/pre-step as a Session memory block (into history) — later rounds ride the history + prefix cache instead of re-rendering every round. forceInject entries render full text (behavioural rules); session entries render as summary titles with id.
2. Dynamic (agent/pre-step listener): retrieves memories matching the current message via HybridSearch, appended at the message tail with a provenance preamble so the model never mistakes them for dialogue. A retrieval gate skips trivial messages; dynamic results exclude the fixed injection set (one memory appears at most once per prompt).

Injected memories render as - [category] title (id) — details expand via the view tool by id.

Search

• KeywordSearch: ASCII words (min 2 chars) + CJK 2-grams, stopword-filtered. Generic words (文件/测试/文档…) count as half a hit; queries with ≥3 tokens need ≥2 weighted hits. No-match → score 0.
• EmbeddingSearch: cosine similarity against stored vectors, minSimilarity relevance gate (default 0.65). Dimension mismatch is detected and logged.
• HybridSearch: Reciprocal Rank Fusion (k=60) over both ranked lists; score-0 entries are filtered before fusion.

Embedding provider: local ollama (qwen3-embedding:0.6b, 1024-dim), overridable via DSH_MEMORY_EMBED_MODEL. Degrades to keyword-only when ollama is unavailable.

Cache & Data Safety

• Process-local cache with mtime invalidation — external edits to memory.yaml are picked up
• Serialized write queue — concurrent appends never lose entries
• Atomic writes (tmp + rename) — a crash never leaves a truncated file
• Corrupt-file quarantine — a parse failure backs up the file and refuses writes instead of silently overwriting

Tools

Tool	Purpose
remember	Save (content / title / category / tags / importance / forceInject / injectLevel / source)
recall	Hybrid search (keyword + embedding) top-N
view	Expand ONE memory's full content by id
list	List memories as titles (category/tag/importance filters, full for content)
forget	Soft-delete (marks retired, keeps history)
pin	Toggle forceInject — pinned memories always appear at session start

Roadmap

v	Capability
v1	keyword + injection + YAML
v2	embedding search (HybridSearch)
v3	dynamic topic injection + provenance labeling
v4	title-first injection + two-level (session/auto) injection
v5	feedback scoring (planned)
v6	multi-scope isolation (planned)