Scripts for BEAT Dataset

March 30, 2022 ยท View on GitHub

Contents

  • train and inference scripts
    • CaMN (ours)
    • End2End (ours)
    • Motion AutoEncoder (for evaluation)
    • data preprocessing
      • load specific number of joints with predefined FPS from bvh
      • build word2vec model
      • cache generation (.lmdb)
  • dataset examples in beat.zip
    • original files to generate cache in train/val/test
    • cache for language_model, pretrained_vae

Train

  1. python == 3.7
  2. build folders like:
    • codes
    • datasets
    • outputs
  3. download the scripts to codes/beat/
  4. extract beat.zip to datasets/beat
  5. run pip install -r requirements.txt in the path ./codes/beat/
  6. run python train.py -c ./configs/camn.yaml for training and inference.
  7. load ./outputs/exp_name/119/res_000_008.bvh into blender to visualize the test results.

Modifiaction

  • train End2End model, add g_name: PoseGenerator in camn.yaml
  • generate data cache from stratch
    • cd ./dataloaders && python bvh2anyjoints.py for motion data
    • cd ./dataloaders && python build_vocab.py for language model
  • remove modalities, e.g., remove facial expressions.
    • set facial_rep: None and facial_f: 0 in camn.yaml
    • python train.py -c ./configs/camn.yaml
    • for semantic-weighted loss, set sem_weighted == False in camn_trainer.py
  • refer to ./utils/config.py for other parameters.

Updated, remove all personal informaiton in scripts.