ReFind

ReFind is an agentic long-term memory retriever packaged for the Agent Memory Leaderboard Add/Search API. It stores the benchmark's raw memory chunks and uses a small planning model to iteratively search a conversation-level BM25 index, preserve relevant evidence, and return contextual memory blocks to the leaderboard's shared answer model.

Paper & related work

📄 Our ReFind paper is now available: When Your Agent Opens the Chat App: Agent-Controlled Search over Raw Chat Logs Rivals Structured Memory.

Also check out our other memory-related benchmark, InMind: Keep It InMind: Benchmarking the Implicit-Association Blind Spot in Agent Memory.

Citation

If you find ReFind useful, please cite our paper:

@misc{li2026refind,
  title         = {When Your Agent Opens the Chat App: Agent-Controlled Search over Raw Chat Logs Rivals Structured Memory},
  author        = {Ruizhe Li and Licheng Zhang and Benfeng Xu and Mingxuan Du and Zheren Fu and Weidong Chen},
  year          = {2026},
  eprint        = {2608.12888},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2608.12888}
}

Method

The service adapts the reported ReFind/NoteTaker retrieval stage to the competition boundary:

1. Synchronous memory ingestion. Every Add payload is committed to SQLite before HTTP 200 is returned. Repeated request_id payloads are idempotent; conflicting reuse is rejected.
2. Conversation-level index. Platform Add chunks are stored raw, then grouped by exact user_id and session_id, ordered by source timestamp/chunk ordinal, and paired into adjacent-turn conversation records at Search time. BM25 uses stop-word removal and Porter-style stemming, then Reciprocal Rank Fusion combines conversation-level and session-level rankings.
3. Iterative retrieval agent. GPT-4o-mini plans up to four search_chatrecord, take_note, or finish_search actions. Internal searches retrieve top 5 records by default.
4. Temporal context. Each result expands to two neighboring chunks on either side, and the planner can apply absolute date filters.
5. Competition response. ReFind returns only ordered evidence. It deliberately omits the paper's second-stage answer generation because the leaderboard passes returned content to a shared answer pipeline.

The no-key bm25 mode is included for local smoke testing and ablations. Competition evaluation of the full method must use RETRIEVAL_MODE=agent with an LLM key; Search fails explicitly if agent mode is selected without one, preventing a silent baseline evaluation.

The original technical report, authorship, preserved components, and every competition-specific change are disclosed in docs/METHOD_CARD.md. The submission configuration uses GPT-4o-mini, as required by the 2026 challenge rules.

A form-ready summary of the academic code-submission route, endpoints, Docker command, and recommended concurrency is available in docs/SUBMISSION.md.

API contract

The recommended endpoints are:

• POST /v1/memories/add
• POST /v1/memories/search
• GET /health

POST /add and POST /search are equivalent compatibility aliases.

Add

{
  "request_id": "eval:run:dataset:conv-0:chunk-0",
  "messages": [
    {
      "role": "user",
      "timestamp": 1704067200000,
      "content": "I adopted a cat named Luna."
    }
  ],
  "user_id": "eval:run:dataset:conv-0",
  "session_id": "eval:run:sample:0"
}

Successful response:

{
  "success": true,
  "request_id": "eval:run:dataset:conv-0:chunk-0",
  "user_id": "eval:run:dataset:conv-0",
  "session_id": "eval:run:sample:0"
}

Search

{
  "query": "What is the name of my cat?",
  "user_id": "eval:run:dataset:conv-0",
  "top_k": 100
}

Successful response:

{
  "data": [
    {
      "id": "mem_...",
      "content": "[2024-01-01T00:00:00Z] USER: I adopted a cat named Luna.",
      "score": 1.25,
      "created_at": "2026-08-07T12:00:00Z"
    }
  ]
}

Results never cross user_id boundaries and never exceed the requested top_k. Multiple Add requests belonging to the same session_id are reconstructed in source order before BM25 indexing; SQLite arrival order is only used when both timestamps and request chunk ordinals are unavailable.

Run with Docker

Build and run the full method with OpenRouter:

docker build -t refind:latest .
docker run --rm \
  -p 8000:8000 \
  -e LLM_API_KEY="your-openrouter-key" \
  -e RETRIEVAL_MODE=agent \
  -v refind-data:/data \
  refind:latest

The container exposes port 8000, stores its database at /data/refind.sqlite3, and runs without endpoint authentication so the competition maintainer can call it directly.

For a key-free local contract check:

docker run --rm -p 8000:8000 -e RETRIEVAL_MODE=bm25 refind:latest
python scripts/smoke_test.py http://127.0.0.1:8000

Configuration

Variable	Default	Purpose
RETRIEVAL_MODE	agent	agent for ReFind or bm25 for the deterministic ablation
LLM_API_KEY	—	OpenAI-compatible provider key; OPENROUTER_API_KEY and OPENAI_API_KEY are also recognized
LLM_BASE_URL	https://openrouter.ai/api/v1	OpenAI-compatible API base URL
LLM_MODEL	openai/gpt-4o-mini	Phase-1 retrieval planner
AGENT_MAX_ITERATIONS	4	Maximum planner actions per Search request
SEARCH_TOP_K	5	Results exposed to the planner per internal search
CONTEXT_WINDOW	2	Neighbor chunks on each side of a hit
SESSION_RRF	true	Fuse conversation and aggregate session BM25 rankings
DATABASE_PATH	./data/refind.sqlite3	SQLite path; the image sets /data/refind.sqlite3
RETENTION_DAYS	30	Persistence window, constrained to 1–30 days

No API keys are committed or written to logs. The service logs only aggregate request statistics, not evaluation memories or questions. Evaluation records older than the configured retention window are automatically deleted.

Local development

Use Python 3.11 or newer:

python -m pip install -r requirements-dev.txt
RETRIEVAL_MODE=bm25 uvicorn app.main:app --host 0.0.0.0 --port 8000
pytest

The OpenAPI document is available at /docs while the service is running.