LIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in Vision-Language-Action Models

📄 Paper  |  🌐 Website  |  🤗 Datasets  |  📦 Assets  |  🧠 Model

🎉 Accepted to ECCV 2026

🔥 News & Updates

• 2026-06-18: LIBERO-Safety was accepted to ECCV 2026.
• 2026-04-23: Open-sourced LeRobot training dataset.
• 2026-03-20: Released code, assets and the pi₀.₅ fine-tuned weights.
• Coming soon: We will open-source the data generation pipeline.

🔥 Overview

This repository provides the official LIBERO-Safety codebase built on top of LIBERO. It introduces a benchmark for physical and semantic safety in Vision-Language-Action (VLA) models, with tasks organized into five suites and three difficulty levels (L0–L2) per suite. A keypose-driven data generation pipeline is used to synthesize large-scale collision-free demonstrations for training and evaluation.

Usage is drop-in compatible with the standard LIBERO workflow: replace your LIBERO checkout with this repository and install with pip install -e .—existing LIBERO-style scripts and configs largely work the same way.

📊 LIBERO-Safety Benchmark

Five task suites (code names → benchmark_name in config)

Suite (paper / website)	benchmark_name (Hydra / get_benchmark)
Affordance-aware grasping	affordance
Human–robot interaction	human_safety
Tabletop spatial avoidance	obstacle_avoidance
Free-space hand–object avoidance	obstacle_avoidance_human
Semantic safety reasoning	reasoning_safety

Each suite defines 5 tasks × 3 levels (L0–L2) in libero/libero/benchmark/vla_safety_task_map.py.

Original LIBERO suites (optional)

The same benchmark registry also includes standard LIBERO splits: libero_10, libero_90, libero_spatial, libero_object, libero_goal.

Evaluated models

To benchmark safety across physical execution and cognitive reasoning, we evaluate 10 representative architectures, grouped into 4 paradigms:

1. Standard VLA models — OpenVLA, OpenVLA-OFT, π₀, and π0.5 (OpenPI).
2. World model (WM)–based VLAs — UniVLA and VLA-JEPA.
3. Dual-system VLA frameworks — GR00T N1.5 and GR00T N1.6 (GR00T).
4. Embodied foundation models — RynnBrain and RoboBrain, used only for Semantic safety reasoning suite.

🛠️ Installation

The workflow mirrors LIBERO. If you already have LIBERO installed, uninstall or remove it first, then use this repo.

git clone https://github.com/LIBERO-SAFETY/LIBERO-Safety.git
cd LIBERO-Safety

Config paths. LIBERO resolves paths via ~/.libero/config.yaml (override with env LIBERO_CONFIG_PATH). On first import, defaults may be created; ensure keys assets, bddl_files, benchmark_root, datasets, init_states point into this repository.

You can inspect the default config logic in libero/libero/__init__.py.

# Editable install (package name remains `libero` for compatibility)
pip install -e .
pip install -r requirements.txt
pip install -r extra_requirements.txt

# Install third-party robosuite dependency
pip install -e third_party/robosuite-1.4

System dependencies (robosuite / rendering). Depending on your setup, you may need packages similar to upstream LIBERO or LIBERO-Plus, e.g.:

sudo apt-get update
sudo apt-get install -y libexpat1 libfontconfig1-dev libpython3-stdlib libmagickwand-dev

Assets. Please download our assets from the LIBERO-Safety, including required objects, textures, and safety-benchmark resources. Please unzip the assets.zip file to LIBERO-Safety/libero/libero/ so that assets are located under libero/libero/assets/. We provide LeRobot-format dataset for training.

📈 Evaluation & leaderboard

• Project page: metrics, figures, and qualitative failure cases are summarized on the website.
• Paper: full protocol and numbers will appear in the camera-ready / arXiv version (update the Paper link at the top when available).

📜 Citation

If you find LIBERO-Safety useful in your research, please cite our paper:

@article{cui2026liberosafety,
  title={LIBERO-Safety: A Comprehensive Benchmark for Physical and Semantic Safety in Vision-Language-Action Models},
  author={Rongxu Cui and Zongzheng Zhang and Jingrui Pang and Haohan Chi and Jinbang Guo and Saining Zhang and Shaoxuan Xie and Xin Jin and Yao Mu and Jiaolong Yang and Guocai Yao and Xianyuan Zhan and Ya-Qin Zhang and Hao Zhao},
  journal={arXiv preprint arXiv:2606.23686},
  year={2026}
}

Acknowledgments

This project builds on LIBERO and follows the documentation style of community forks such as LIBERO-Plus. We thank the LIBERO team and the open-source robotics community.