Contributing
September 19, 2026 ยท View on GitHub
Adding an adapter
- Read
src/invalidate/adapters/base.py.InMemoryAdapteris the template; theAdapterprotocol is three methods (pull,flag,delete) plus optionalinsert. - Install the host SDK into
.venvonly to read its source. Mirror the exact signatures and return shapes, and cite the version and file:line in the module docstring. Import the SDK lazily or not at all:import invalidate.adaptersmust stay dependency-free. pull()is read-only and returns verbatim text with the host's own ids. If the host has no stable ids, content-address them (sha1(normalized text)[:10]) as the Markdown adapter does.flag()writesreason.as_metadata()if the host has metadata, orreason.line()if it only holds text (Letta writes a sidecar note; Markdown writes an HTML comment).Status.ACTIVEmeans undo. Never rewrite the stored text.delete()may be a soft delete if the host keeps history (Graphiti setsinvalid_at).insert()stores the event text verbatim (Mem0:add(..., infer=False)). If the host can only store through its own extraction, return the id anyway and document that the stored form is not verbatim. Make sure a subsequentpull()returns the successor, or the nextsync()will forget it.- Tests:
tests/test_adapters_<host>.pyagainst a faithful fake of the client. Cover sync, flag, delete, insert, filter, error paths, and every documented limit. Run the sharedFakeJudgefromtests/conftest.py. If the host runs in-process (Chroma, LangGraph), test against the real thing. - A live script under
scripts/live_<host>.pyand a pagedocs/adapters/<host>.md: install, 15-line usage, what flag/delete/insert do in that host, real limits. - Add a row to the table in
ADAPTERS.mdand an extra inpyproject.toml.
Adding eval cases
evals/cases.py. Every case needs a category, a memory, an event, an expected label, and a
lenient list when more than one label is defensible. Include hard negatives whose surface form
matches the category but whose label is the opposite. Never copy a case string into the question
criteria in src/invalidate/questions.py; paraphrase with different entities.
Changing the questions or the policy
Run python evals/run_eval.py --out evals/results/<name>.json before and after, and paste both
summaries in the PR. A change that raises strict accuracy but adds a false invalidation is a
regression.
Growth
The layer wins by being in front of every store. The order that matters: the stores with the most agents (Mem0, LangGraph, plain vector DBs), then the ones with the strongest opinions (Letta, Graphiti), then event sources that no memory product ingests (Slack, GitHub, Linear webhooks), then the surfaces where developers already keep memory in files (Claude Code, Cursor rules, AGENTS.md). Each adapter should ship with a live-verified script and a one-line demo that runs on the contributor's own data.