\[ICCV 2025\] STaR: Seamless Spatial-Temporal Aware Motion Retargeting with Penetration and Consistency Constraints

This is the code for the ICCV 2025 paper "STaR: Seamless Spatial-Temporal Aware Motion Retargeting with Penetration and Consistency Constraints", by Xiaohang Yang, et al.

Neural motion retargeting model, STaR, is designed for a balance between: (1) motion semantics preservation, (2) physical plausibility, and (3) temporal consistency.

🚀 Update: The full code will be released soon—stay tuned! 👀

• test code, with network weight and a processed test set;
• inference code, with a PyTorch3D-based rendering function;
• data preparation process & pre-processed data;
• release a skel2mesh motion retargeting sub-model based on STaR, which can be used for retargeting motion generation results to new characters;

Environment Setup

1. Creete Conda environment

We tested with Python 3.9, CUDA 11.8.

conda create python=3.9 -n star
conda activate star

2. Install main dependencies

• Install PyTorch. We have tested on PyTorch 2.3.1.

conda install pytorch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 pytorch-cuda=11.8 -c pytorch -c nvidia

• Install PyTorch3D (for visualisation only).

  Build from source code or refer to this link for pre-built wheels that matches your environments.

3. Install other packages

pip install numpy==1.23.3 tqdm wandb scipy matplotlib==3.4.2 ninja
cd ./submodules
pip install ./pointnet2_ops_lib/.

Data Preparation

The Easy Way

Download the processed train / test / inference set from here. Put all content under ./datasets/mixamo/.

This will take more space.

Full Pipeline

We use scripts to download, match motion instances and create datesets.

1. Download the motion data from the Adobe Mixamo. The JavaScript can be pasted and used in browsers like Chrome. Download one sample motion before running the script to fix the motion parameters (like wo_skin and FPS).

./preprocess/download.js

The downloaded files contain all motions and a record txtfile.

2. Rename the files, multiple motion clips have the same motion names. We match the motion clips, and sample train / test sets. Change the folder_path and other parameters before running.

python ./preprocess/1_rename_files.py

3. Transfer the FBX files to BVH files.

python ./preprocess/2_fbx2bvh.py

4. Get the quaternion from the BVH files.

python ./preprocess/3_preprocess_q.py

5. Load the train / test pairs, re-check if the motion clips are matched in the step 2.

python ./preprocess/4_load_train_test_json.py

6. Extract the shape information. Blender Python API is required for this part.

blender -b -P ./preprocess/5_extract_shape.py

Folder Organization

Here is an example folder structure after running the full data preparation pipeline:

STaR/
├── datasets/
│   └── mixamo/
│       ├── encoding_model_weights/
│       ├── inference
│           ├── inference_data.npy    # option 1, downloaded from Google Drive
│           ├── inference_q           # option 2, generated by the preprocess scripts
│           └── inference.json
│       ├── scale_info
│       ├── shape
│       ├── stats
│       ├── test
│           ├── test_data.npy         # option 1, downloaded from Google Drive
│           ├── test_q                # option 2, generated by the preprocess scripts
│           └── test.json
│       ├── test_shape
│       ├── texture_datapath
│       └── train
│           ├── train_data.npy        # option 1, downloaded from Google Drive (not uploaded yet)
│           ├── train_q               # option 2, generated by the preprocess scripts
│           └── train.json
│   ├── other_dataset/
│   ├── train_set.py
│   ├── test_set.py
│   └── inference_set.py

Note: Some folders (like shape/, render_dir/, work_dir/) may be generated after running respective scripts, and may initially be empty or missing.

Test

We have include the weights in this repository, the weight path is ./work_dir/paper_version/retarget_model.pt.

python test.py --config ./config/test_config.yaml

The result will be save in ./work_dir. The curvature metric is for each joint, you can get the average curvature for each limb.

Inference and Visualisation

We use PyTorch3D for rendering, no Blender Python API API requirements.

python inference.py --config ./config/inference_config.yaml

All rendered videos will be saved in ./render_dir/.../. Edit the parameters, render_mode and ./render_direction, for different results. For example, changing render_mode from default to rotate, you can get these two different result:

Train

We split the trainset into train split and validation split.

python train.py --config ./config/train_config.yaml

Skel2Mesh Motion Retargeting

(With In Progress)

Acknowledgments

The code is partially built based on R2ET.

The dataset is taken from Adobe Mixamo.

The Chamfer Distance is adapted from ChamferDistancePytorch.

The shape encoder and PointNet2_lib is adapted from PCT.

Thanks to all of them.

Citation

If you find this work helpful, please consider citing it as follows:

@inproceedings{yang2025star,
  title={STaR: Seamless Spatial-Temporal Aware Motion Retargeting with Penetration and Consistency Constraints},
  author={Yang, Xiaohang and Wang, Qing and Yang, Jiahao and Slabaugh, Gregory and Yuan, Shanxin},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={12947--12955},
  year={2025}
}

Or

@article{yang2025star,
  title={STaR: Seamless Spatial-Temporal Aware Motion Retargeting with Penetration and Consistency Constraints},
  author={Yang, Xiaohang and Wang, Qing and Yang, Jiahao and Slabaugh, Gregory and Yuan, Shanxin},
  journal={arXiv e-prints},
  pages={arXiv--2504},
  year={2025}
}