This is the official repository for VersatileMotion.

📌 Overview

Recent 3D human motion modeling largely follows two paradigms: continuous diffusion, and discrete motion tokenization followed by autoregressive (AR) Transformers. Prior works showed that the discrete–AR route can unify generation and comprehension, yet most follow-ups remain confined to text–motion pairs and single-human settings. VersatileMotion studies the versatility limits of the discrete–AR paradigm as a unified solution for motion.

We first revisit motion tokenization and introduce VerMoVQ, a causal spatio-temporal motion VQ-VAE that robustly quantizes long, detailed sequences. Although trained on single-human data, VerMoVQ preserves world-space root trajectories, enabling straightforward application to multi-human tokenization. On top of these tokens, we add VerMo Refiner, a refiner based on flow-matching DiT operating in motion-latent space, which improves temporal smoothness and local details without altering the discrete interface.

Using the resulting motion tokens, we train a single AR Transformer, VersatileMotion, over a unified vocabulary of motion, text, and audio to accommodate diverse tasks and conditioning combinations. To support training and evaluation, we curate MotionHub, a ~400K-clip corpus with paired textual and audio annotations. Across nine motion generation and understanding benchmarks, our models consistently surpass strong task-specific baselines on seven, and enable flexible cross-modal behaviors (e.g., text+audio → motion).

📦 MotionHub Dataset (Released)

MotionHub is a large-scale motion corpus curated to support training and evaluation of versatile motion models. It contains ~400K motion clips spanning diverse styles and domains. MotionHub was built to facilitate research in both motion generation and motion understanding tasks by providing:

• Unified format: All motion sequences are converted to Blender-ready SMPLX-55 npz files with consistent joint definitions.
• Diverse sources: Includes single-person motion, two-person interactions, dance, and speaking actions assembled from multiple established datasets.
• Multi-modality data pairs: Besides common text–motion pairs, we also provide multi-modal pairs such as text–audio–motion, including a large amount of manually annotated text.

The MotionHub dataset is publicly available at:
https://github.com/ZeyuLing/Motionhub

🧩 TODO

• Release VerMoVQ & VerMo Refiner implementation — robust motion quantization and refinement tools underpinning VersatileMotion.
• Release VersatileMotion model implementation — AR Transformer and training code supporting unified motion/text/audio tasks.