LHM_Track: Build Your LHM Training Dataset

April 8, 2025 · View on GitHub

Lingteng Qiu*, Xiaodong Gu*, Peihao Li*, Qi Zuo*, Weichao Shen, Junfei Zhang, Kejie Qiu, Weihao Yuan
Guanying Chen+, Zilong Dong+, Liefeng Bo

Tongyi Lab, Alibaba Group

Code Project Website arXiv Paper HuggingFace ModelScope Apache License

Description

LHM_Track is a video processing pipeline for building LHM training dataset. It generates multiple labels from human centric video, such as segmentation, 2D keypoints, SMPL-X parameters and FLAME parameters.

If you find our work useful for you, please ⭐ star LHM and LHM_Track repositories to help us reach more people.

📢 Latest Updates

[April 8, 2025] Release the pipeline for building LHM training dataset.

🚀 Getting Started

Environment Setup

Clone the repository.

git clone https://github.com/aigc3d/LHM_Track.git
cd LHM_Track

The environment is not same as the LHM repo. Please create a new conda environment or python venv.

# create new environment
conda create -n lhm_track python=3.10
conda activate lhm_track
# or
# python -m venv lhm_track
# source lhm_track/bin/activate

# cuda 12.1
sh ./install_cu121.sh

# cuda 11.8
sh ./install_cu118.sh

The installation has been tested with python3.10, CUDA 12.1 or CUDA 11.8.

Download Prior Model Weights

Download basic model weights. If you've downloaded them in LHM repo, you can copy/link the pretrained_models folder into this repo.

# Download prior model weights
wget https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/aigc3d/data/LHM/LHM_prior_model.tar 
tar -xvf LHM_prior_model.tar

Download extra model weights.

# Download extra model weights
wget https://virutalbuy-public.oss-cn-hangzhou.aliyuncs.com/share/aigc3d/data/LHM/LHM_track_model.tar 
tar -xvf LHM_track_model.tar

🏃 Inference Pipeline

python track_video.py --video_path ${VIDEO_PATH} --output_path ${OUTPUT_PATH}

Citation

@inproceedings{qiu2025LHM,
  title={LHM: Large Animatable Human Reconstruction Model from a Single Image in Seconds},
  author={Lingteng Qiu and Xiaodong Gu and Peihao Li  and Qi Zuo
     and Weichao Shen and Junfei Zhang and Kejie Qiu and Weihao Yuan
     and Guanying Chen and Zilong Dong and Liefeng Bo 
    },
  booktitle={arXiv preprint arXiv:2503.10625},
  year={2025}
}

Acknowledgement

This work is built on many amazing research works and open-source projects:

Thanks for their excellent works and great contribution to 3D generation and 3D digital human area.