Isaac-Humanoid-Benchmark

A diverse humanoid benchmark based on Isaac Lab and a testbed for manipulation policy generalization.

Overview

This is the repo of simulation benchmark for our work:

EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos

Ruihan Yang^1*, Qinxi Yu^2*, Yecheng Wu^3,4, Rui Yan¹, Borui Li¹, An-Chieh Cheng¹, Xueyan Zou¹, Yunhao Fang¹, Xuxin Cheng¹, Ri-Zhao Qiu¹, Hongxu Yin⁴, Sifei Liu⁴, Song Han^3,4, Yao Lu⁴, Xiaolong Wang¹

¹UC San Diego / ²UIUC / ³MIT / ⁴NVIDIA

Project Page / Arxiv / Training code and eval

For training code and eval code, follow: https://github.com/RchalYang/EgoVLA_Release

Getting Started

Notice!

We are using IsaacLab version 1.2.0 and IsaacSim 4.2.0 for EgoVLA data collection and evaluation. As we observe large change of physics behaviours in later IsaacLab & IsaacSim version, please consider downgrade versions to reproduce results.

Installation

1. Install IsaacLab
   Follow the IsaacLab Local Installation Guide.

2. Clone Ego_Humanoid_Manipulation_Benchmark

   Note: The asset data are hosted at Huggingface due to quota restriction, check https://huggingface.co/EgoVLA for asset and collected data.

   !!Note: If you don't have git lfs install (required for downloading assets), run

   git lfs install
   

   !!Note: Since we are hosting LFS files on hugging face repo as a submodule, you might need to set up authentication with your hugging face account.

   Clone repo & assets:

   git clone --recurse-submodules https://github.com/quincy-u/Ego_Humanoid_Manipulation_Benchmark.git
   

3. Install the Humanoid Tasks Extension

   cd Ego_Humanoid_Manipulation_Benchmark && ${IsaacLab_PATH}/isaaclab.sh -p -m pip install -e source/extensions/humanoid.tasks
   

4. Adjust Rendering and Physics (Recommended)
   Replace the following files in your IsaacLab installation with the versions from this repo for adjusted rendering and physics:

   • source/apps/isaaclab.python.kit
   • source/apps/isaaclab.python.rendering.kit
     (Do the same for headless apps if needed.)

   Note: In newer versions of IsaacLab, these files are located under apps/ instead of source/apps/ and may not be directly replaceable. In that case, update the corresponding settings manually as needed.

---

Usage

To run an environment:

${IsaacLab_PATH}/isaaclab.sh -p scripts/<agent_file> --task <environment_name> --num_envs <number_of_environments> --enable_cameras

Example:
Run the Humanoid-Push-Box-v0 environment with random actions and 4 parallel environments:

${IsaacLab_PATH}/isaaclab.sh -p scripts/random_agent.py --task Humanoid-Push-Box-v0 --num_envs=4 --enable_cameras

• <agent_file>: Python file specifying the agent (e.g., random_agent.py)
• <environment_name>: Name of the environment (see table below)

Env Config:
You can override configs directly before gym.make(args_cli.task, cfg=env_cfg).
Example: env_cfg.room_idx = 2 This changes the background room. See the full list of configurable parameters below.

configurable parameters

#	Variable Name	Description	Type
1	episode_length_s	Episode length in seconds	float
2	decimation	Robot control decimation	int
3	action_scale	Action scaling factor	float
4	spawn_table	Spawn a table in the environment	bool
5	spawn_background	Spawn a background room in the environment	bool
6	room_idx	Index of background room to use (1–5)	int
7	table_idx	Index of table to use (1–3)	int
8	seed	Random seed used by Isaac Sim	int
9	randomize	Whether to randomize the environment	bool
10	randomize_range	Randomization range factor	float
11	randomize_idx	< 0: non-reproducible randomization. Otherwise, reproducible randomization given fixed index	int

For the details of randomization implementation, check each the _reset_idx() implementation of each env file.

Available Environments

#	Environment Name
1	Humanoid-Close-Drawer-v0
2	Humanoid-Open-Drawer-v0
3	Humanoid-Flip-Mug-v0
4	Humanoid-Open-Laptop-v0
5	Humanoid-Pour-Balls-v0
6	Humanoid-Push-Box-v0
7	Humanoid-Stack-Can-v0
8	Humanoid-Stack-Can-Into-Drawer-v0
9	Humanoid-Unload-Cans-v0
10	Humanoid-Insert-Cans-v0
11	Humanoid-Sort-Cans-v0
12	Humanoid-Insert-And-Unload-Cans-v0

---