🤖 HumanoidGen: Data Generation for Bimanual Dexterous Manipulation via LLM Reasoning

November 6, 2025 · View on GitHub

Zhi Jing1,2, Siyuan Yang3,2, Jicong Ao2, Ting Xiao4, Yugang Jiang1, Chenjia Bai✉ 2
1Fudan University † 2Institute of Artificial Intelligence (TeleAI), China Telecom †
3University of Science and Technology of China 4East China University of Science and Technology
†Equally leading organizations ✉ Corresponding Author

🔥 Homepage 📄 Paper ⛁ Dataset 🤗 Model

🔥 Updates

[2025-09-18] 🔥🔥🔥 HumanoidGen gets accepted to NeurIPS 2025: Paper

⚙️ Setup Environment

Requirements

  • Supported platform: Linux (Ubuntu 20.04)
  • Python 3.9

Installation

1. Clone the repository

git clone git@github.com:TeleHuman/HumanoidGen.git
cd HumanoidGen

2. Create & Build conda env

conda create --name humanoidgen python=3.9 -y
conda activate humanoidgen
pip install -r requirements.txt
pip install -e .

After installing the mplib library, change the parameter n_init_qposfrom the default value of 20 to 50 in mplib/planner.py. To locate the file path, you can use the following command within the humanoidgen conda environment:

python -c "import mplib; print(mplib.planner.__file__)"

3. Install pytorch3d & dp & dp3

Install pytorch3d:

cd third_party/pytorch3d_simplified && pip install -e . && cd ../..

Install dp3:

cd humanoidgen/policy/3D-Diffusion-Policy/3D-Diffusion-Policy && pip install -e . && cd ../../../..

Install dp:

cd humanoidgen/policy/Diffusion-Policy && pip install -e . && cd ../../..

4. Download Assets

The assets are provided in our datasets lab datasets. Download the files assets.zip and table_assets.zip, extract them to the humanoidgen and scene_builder/table directories respectively, and name both extracted folders as assets.

🚀 Getting Started

1. Change Execution Path

cd humanoidgen

2. Show Scene & Run Task

In this project, we provide standard scenes and execution code for 20 tasks, which can be quickly run using the script files below:

bash scripts/run_scene.sh
bash scripts/run_solve.sh

Additionally, you can specify main parameters by directly modifying the shell script files or by using the following commands:

python process/run_scene.py -env blocks_stack_easy -render False
python process/run_solve.py -env blocks_stack_easy -solve blocks_stack_easy -render False

To configure additional parameters, edit the config files config_run_scene.yml and config_run_solve.yml.

3. Collect & Visualize Data

To collect data, open the configuration file config_run_solve.yml and set record_data to true. Then run the following command (example for ‘block_handover‘ task):

python process/run_solve.py -env block_handover -solve block_handover -render False

The datasets are generated in datasets folder and can be visualized using the following command:

python process/show_datasets.py

The visualization parameters are set in the configuration file config_show_datasets.yml.

4. Train & Deploy Policy

Firstly, pre-process the generated datasets for training policy.

python process/pkl2zarr.py

The dataset path, policy model, and additional parameters are set in the configuration file config_pkl2zarr.yml.

For DP and DP3 policy training and evaluation, we also provide the corresponding the datasets and models.

Dp3 policy train (Configuration File Location):

bash scripts/train.sh dp3

Dp policy train (Configuration File Location):

bash scripts/train.sh dp

Dp3 policy eval (Configuration File Location):

bash scripts/eval_dp3.sh

Dp policy eval (Configuration File Location):

bash scripts/eval_dp.sh

5. Generate Task Execution Code

This project supports two generation methods: direct generation and using MCTS. The execution commands are as follows:

bash scripts/run_generate_solve.sh block_handover 5
bash scripts/run_generate_solve_with_mcts.sh blocks_stack_hard_mcts 5

The first argument specifies the name of the generated task, and the second argument specifies the number of parallel threads to run. To interrupt the generation process, run:

bash scripts/kill_all_generate_processes.sh

📦 Code to be released

  • Scene scaling with Robocasa
  • Scene generation
  • Generation of additional tasks (both MCTS and non-MCTS)

🔖 Citation

If you find our work helpful, please cite:

@article{jing2025humanoidgen,
  title={HumanoidGen: Data Generation for Bimanual Dexterous Manipulation via LLM Reasoning},
  author={Jing, Zhi and Yang, Siyuan and Ao, Jicong and Xiao, Ting and Jiang, Yugang and Bai, Chenjia},
  journal={arXiv preprint arXiv:2507.00833},
  year={2025}
}

📄 License

This codebase is under CC BY-NC 4.0 license. You may not use the material for commercial purposes, e.g., to make demos to advertise your commercial products.

Acknowledgements

📬 Contact

Feel free to contact us!