Text2HOI
March 6, 2025 · View on GitHub
Official code of Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interaction in CVPR 2024
Arxiv paper. / Project page.
Data
| Dataset | Resource |
|---|---|
| H2O | Download |
| GRAB | Download |
| ARCTIC | Download |
| ARCTIC Text Description | Download |
| MANO | Download |
| Object Pickle File | Download |
Preprocessing GRAB object
GRAB objects have so many vertiecs. So we reduce the number of vertices to 4,000.
python preprocessing_grab_object.py
Folder Tree for Demo
data
├─ h2o
│ ├─ obj.pkl
│ └─ object
│ ├─ book
│ ├─ cappuccino
│ ├─ chips
│ ├─ cocoa
│ ├─ espresso
│ ├─ lotion
│ ├─ milk
│ └─ spray
|
├─ grab
│ ├─ obj.pkl
│ ├─ processed_object_meshes
│ └─ contact_meshes
│ ├─ airplane.ply
│ ├─ alarmclock.ply
│ ├─ apple.ply
│ ├─ banana.ply
│ ...
|
├─ arctic
│ ├─ obj.pkl
│ └─ downloads/data/meta/object_vtemplates
│ ├─ box
│ ├─ capsulemachine
│ ├─ espressomachine
│ ├─ ketchup
│ ...
│
└─ mano
└─ mano_v1_2
└─ models
├─ info.txt
├─ LICENSE.txt
├─ MANO_LEFT.pkl
└─ MANO_RIGHT.pkl
Checkpoints
checkpoints
├─ h2o
├─ grab
└─ arctic
Installation
source scripts/install.sh
Demo
source scripts/demo.sh
Train
Folder Tree for Preprocessing
data
├─ h2o
│ ├─ object
│ ├─ subject1
│ ├─ subject2
│ └─ subject3
|
├─ grab
│ ├─ contact_meshes
│ ├─ processed_object_meshes
│ ├─ s1
│ ├─ s2
│ ├─ ...
│ └─ s10
│
├─ arctic
│ └─ downloads
│ └─ data
│ ├─ raw_seqs
| ├─ description # (Download available in the 'Data - Arctic' section above or in the 'For Future Work' section below.)
│ └─ meta
│ └─ object_vtemplates
│
└─ mano
└─ mano_v1_2
└─ models
├─ info.txt
├─ LICENSE.txt
├─ MANO_LEFT.pkl
└─ MANO_RIGHT.pkl
Data preparation (preprocessing)
python preprocessing.py
Alternatively, you can download the preprocessed data files, Download.
data
├─ h2o
│ ├─ balance_weights.pkl
│ ├─ data.npz
│ ├─ obj.pkl
│ ├─ text.json
│ ├─ text_count.json
│ └─ text_length.json
|
├─ grab
│ ├─ balance_weights.pkl
│ ├─ data.npz
│ ├─ obj.pkl
│ ├─ text.json
│ ├─ text_count.json
│ └─ text_length.json
│
└─ arctic
├─ balance_weights.pkl
├─ data.npz
├─ obj.pkl
├─ text.json
├─ text_count.json
└─ text_length.json
Train Text2HOI
source scripts/train/train_texthom.sh
Train Refiner
source scripts/train/train_refiner.sh