SUMER: Goal-Directed Search Outperforms Goal-Agnostic Memory Compression in Long-Context Memory Tasks

Yicong Zheng, Kevin L. McKee, Thomas Miconi, Zacharie Bugaud, Mick van Gelderen, Jed McCaleb

Astera Institute, Emeryville, CA, USA

Contact: alan@astera.org

---

TL;DR

(1) Simple goal-directed agentic search outperforms goal-agnostic compression-based methods (or episodic/semantic/etc. memory frameworks) in CURRENT long-context memory tasks.

(2) We need new eval frameworks for probing truly reusable and generalizable memory in LLM agents.

Overview

Existing memory frameworks for long-context tasks focus on finding optimal compression algorithms, building human design bias into systems rather than finding general solutions. SUMER (Search in Uncompressed Memory via Experience Replay) demonstrates that simple goal-directed search applied to raw, uncompressed data outperforms sophisticated compression-based methods.

Using reinforcement learning with verifiable rewards (RLVR), SUMER trains LLM agents to autonomously use basic search tools (semantic + keyword search) on conversational memory, achieving state-of-the-art performance on the LoCoMo benchmark with a 43% improvement over the prior best method.

Key Contributions

• RLVR Framework: End-to-end reinforcement learning approach for long-context memory tasks with verifiable reward signals based on downstream question-answering accuracy
• Search Over Compression: Simple search tools on uncompressed memory outperform human-designed compression algorithms by nearly doubling judge accuracy (33.70 → 66.79)
• State-of-the-Art Results: 43% gain over prior SOTA on LoCoMo benchmark across diverse question types (multi-hop, temporal, open-domain, single-hop)
• New Paradigm: Calls for more dynamic, autonomously scalable benchmarks that resist human design bias

Results

SUMER achieves the following metrics on the LoCoMo benchmark using: (1) Qwen-2.5-7B-Instruct as the main agent model, (2) Qwen3-Embedding-0.6B as the embedding model, and (3) gpt-oss-120b as the judge model:

• Overall Performance: 48.65 F1 / 43.44 BLEU-1 / 66.79 Judge Score
• 43% improvement over previous state-of-the-art
• Nearly doubles judge accuracy vs. strongest compression baseline (MemMachine: 33.70 → 66.79)
• Trained on just 1 conversation, validated on 9 held-out conversations

Method

The Compression Problem

Figure 1: Compression-based approaches (left) use predefined CRUD operations that can discard information crucial for future queries. SUMER (right) maintains raw memory and uses learned search strategies.

SUMER Training Loop

Figure 2: SUMER uses reinforcement learning to train agents that autonomously search and retrieve from uncompressed conversational memory. The reward signal is derived from downstream task performance (question answering accuracy).

Core Components:

• Base Model: Qwen2.5-7B-Instruct
• Training Algorithm: GRPO (Group Relative Policy Optimization) with multi-turn masking
• Search Tools: Semantic search (RAG) + keyword search on raw conversation history
• Reward Function: R = J(y_pred, y_gold) × F1
  • Binary LLM-as-judge correctness (using gpt-oss-120b)
  • F1 score between prediction and ground truth
  • -1 penalty if no answer submitted

Repository Structure

.
├── sumer/           # Main SUMER implementation
│   ├── src/         # Core code 
│   └── verl/        # RL training framework
└── baselines/       # Baseline method implementations
    ├── mem0/        # RAG, Full Context, Langmem, and Mem0
    ├── MemMachine/  # MemMachine baseline
    └── A-mem/       # A-MEM baseline

SUMER Implementation

The main SUMER implementation is located in the sumer/ directory.

For detailed setup instructions, training procedures, and architecture details, see sumer/README.md.

Baseline Implementations

This repository includes implementations of the following baseline methods for comparison:

• RAG
• Full Context
• Langmem
• A-MEM
• Mem0
• MemMachine

All baseline implementations have been adapted to use local LLMs. See baselines/README.md for structure and instructions.

Citation

If you use this code in your research, please cite our paper:

@article{zheng2025sumer,
  title={Goal-Directed Search Outperforms Goal-Agnostic Memory Compression in Long-Context Memory Tasks},
  author={Zheng, Yicong and McKee, Kevin L. and Miconi, Thomas and Bugaud, Zacharie and van Gelderen, Mick and McCaleb, Jed},
  journal={arXiv preprint arXiv:2511.21726},
  year={2025},
  url={https://arxiv.org/abs/2511.21726}
}

Acknowledgments

This work builds on the verl framework from ByteDance Seed Team and uses the LoCoMo benchmark dataset. Special thanks to the verl and SGLang communities for their support.

Contact

For questions or issues, please open an issue on this repository or contact the authors at the emails listed above.