📰 Updates

July 14, 2026 · View on GitHub

RMBench: Memory-Dependent Manipulation Benchmark

RMBench: Memory-Dependent Robotic Manipulation Benchmark with Insights into Policy Design. Under Review, PDF | arXiv | Website | Join our Community 🔥

Tianxing Chen*, Yuran Wang*, Mingleyang Li*, Yan Qin*, Hao Shi, Zixuan Li, Yifan Hu, Yingsheng Zhang, Kaixuan Wang, Yue Chen, Hongcheng Wang, Renjing Xu, Ruihai Wu, Yao Mu, Yaodong Yang, Hao Dong†, Ping Luo†

📰 Updates

2026.07.14 — Since the previously trained Mem-0 checkpoints were not backed up before our development machine was recycled, we have re-organized the training and now publicly release the retrained model weights:

  • M(1) tasks: due to limited computational resources, all M1 tasks were trained jointly into a single multi-task m1_mix model. The complete model, the processed m1_mix dataset, training/inference configs, and all evaluation logs and videos are available at qiuly/Mem-0-m1mix-RMBench and qiuly/Mem-0-m1mix-dataset-RMBench.
  • M(n) tasks: per-task execution-module checkpoints for battery_try, blocks_ranking_try, cover_blocks and press_button, together with per-task normalization stats and evaluation results, are available at qiuly/Mem-0-mn-RMBench.

Detailed evaluation results can be found in the Hugging Face model cards above.

🧑🏻‍💻 RMBench Usage

This project is built upon RoboTwin 2.0, and you can seamlessly transfer your policy code between the two projects.

1. Installation

First, prepare a conda environment.

conda create -n RMBench python=3.10 -y
conda activate RMBench

RMBench Repo: https://github.com/RoboTwin-Platform/RMBench

git clone https://github.com/RoboTwin-Platform/RMBench.git

Then, run script/_install.sh to install basic conda envs and CuRobo:

bash script/_install.sh

2. Download Assets

To download the assets, run the following command. If you encounter any rate-limit issues, please log in to your Hugging Face account by running huggingface-cli login:

bash script/_download_assets.sh

3. Download Data

Please run the following command to download all data.

bash script/_download_data.sh
If you need to collect the data (we actually recommend downloading it directly)

In RMBench, we always use demo_clean setting.

Running the following command will first search for a random seed for the target collection quantity, and then replay the seed to collect data.

Please strictly follow our tutorial in RoboTwin 2.0 Doc - Collect Data.

bash collect_data.sh ${task_name} ${task_config} ${gpu_id}
# Example: bash collect_data.sh cover_blocks demo_clean 0

4. Run Policies

  1. Mem-0 (ours): See Mem-0 Document
  2. DP: See DP Document
  3. ACT: See ACT Document
  4. Pi 0.5: See Pi 0.5 Document
  5. X-VLA: See X-VLA Document
  6. Other Policies (Pi0, RDT, etc): See Document and See Folder
  7. Configure your policy: See Tutorial Here

👍 Citations

If you find our work useful, please consider citing:

@article{chen2026rmbench,
  title={RMBench: Memory-Dependent Robotic Manipulation Benchmark with Insights into Policy Design},
  author={Chen, Tianxing and Wang, Yuran and Li, Mingleyang and Qin, Yan and Shi, Hao and Li, Zixuan and Hu, Yifan and Zhang, Yingsheng and Wang, Kaixuan and Chen, Yue and others},
  journal={arXiv preprint arXiv:2603.01229},
  year={2026}
}

🏷️ License

This repository is released under the MIT license. See LICENSE for additional details.