HiLo: Detailed and Robust 3D Clothed Human Reconstruction with High-and Low-Frequency Information of Parametric Models

Yifan Yang · Dong Liu · Shuhai Zhang · Zeshuai Deng . Zixiong Huang . Mingkui Tan

CVPR 2024

Video demo	In-the-wild reconstruction w/ challenging poses and loose cloth
Comparison with SOTAs	sketch to 3D clothed human

Table of Contents

1. Introduction to HiLo
2. Running Demo
3. Training and testing
4. Citation

Introduction-to-CR-NeRF

Pipeline of HiLo

• If you want to Train & Evaluate, please check installation.md to prepare environment, required models and extra data. Please check dataset.md to prepare THuman2.0 and CAPE dataset, see Training and testing to train and benchmark HiLo using the prepared datasets.

• If you want to Running Demo, please see Running Demo.

Giving a RGB image of clothed human, with our HiLo, you will get:

• image:
  • with the normals of smpl and cloth
• mesh:
  • with the 3d objects of smpl, reconstructed and refined cloth
• video:
  • showing the reconstructed human from all angles

Running Demo

#Set $in_dir, $out_dir and cuda devices in command/infer.sh
bash command/infer.sh

The reconstructed results (mesh, image, video) will be in path "{$out_dir}".

Training and testing

If you want to train and test the model

#Set experiment name and cuda devices in train_and_test.sh 
bash command/train_and_test.sh

If you only want to test the model

#Set the experiment name to match the training name, and set cuda devices in test_only.sh  
bash command/test_only.sh

Citation

@inproceedings{yang2024hilo,
  title={HiLo: Detailed and Robust 3D Clothed Human Reconstruction with High-and Low-Frequency Information of Parametric Models},
  author={Yang, Yifan and Liu, Dong and Zhang, Shuhai and Deng, Zeshuai and Huang, Zixiong and Tan, Mingkui},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={10671--10681},
  year={2024}
}

Acknowledgments

Here are some great resources we benefit from:

• ICON
• ECON
• THuman2.0