This project provides a thorough summary of the latest advancements in the field of 2D digital human motion video generation, covering papers, datasets, and code repositories.

The repository is organized into three main conditions: Vision-driven, Text-driven, and Audio-driven, while also considering LLM Planning Papers.

Unlike previous summaries, this project clearly outlines the five key stages in the field of digital human video generation:

🌑 Stage 1: Input Phase. Clarifying the driving source (Vision, Text, Audio) and driving region (Part, Holistic), where "Part" mainly refers to the face;

🌒 Stage 2: Motion planning Phase. Most work involves feature mapping to learn motion mappings, while a few works use large language models (LLMs) for motion planning;

🌓 Stage 3: Motion Video Generation Phase;

🌔 Stage 4: Video Refinement Phase, focusing on optimizing specific parts such as the face, lips, teeth, and hands;

🌕 Stage 5: Acceleration Phase, aiming to speed up training and deployment inference as much as possible, with the goal of achieving real-time output.

🎉 We welcome everyone to contribute your research and submit PRs to collectively advance the technology of human motion video generation.

If you have any questions, feel free to contact us at (WinniyGD@outlook.com), and we will respond as soon as possible. Additionally, we warmly welcome new members from related fields to join us, learn together, and make endless progress!

本项目认真总结了👍2D数字人动作视频生成👏相关领域的最新进展，包括论文、数据集和代码库。

Repo以 Vision-driven、Text-driven、Audio-driven 三大方向作以总结，同时考虑 LLM Planning 前沿论文。

分类时，我们定义Audio>Text>Vision优先级，当出现文本不出现音频时，归纳为Text-Driven方法，当文本音频同时出现时，归纳为Audio-Driven方法，以此类推。

区别于以往的总结，项目明确总结了数字人视频生成领域的五大阶段：

🌑 第1阶段 明确驱动源（Vision、Text、Audio）与驱动区域（Part、Holistic），其中Part主要以脸部为主；

🌒 第2阶段 动作规划阶段，大多数工作以特征Mapping学习动作映射，少部分工作以大语言模型LLMs进行动作规划；

🌓 第3阶段 人体视频生成，大部分工作以Diffusion Models为基础，少部分工作以Transformer为基础；

🌔 第4阶段 视频优化阶段，针对脸部、嘴唇、牙齿、手部单独做Refinement优化；

🌕 第5阶段 加速输出阶段，尽可能地加速训练与部署推理，目标Real-Time实时输出。

🔑本项目由六位核心成员全力推进：

- 薛海威（清华大学，负责人）
- 罗向阳（清华大学）
- 胡璋昊（爱丁堡大学）
- 张鑫（西安交通大学）
- 向迅之（中国科学院大学）
- 戴语琴（南京理工大学）

💖核心综述由以下老师全力支持并悉心指导：

- 刘健庄老师（中国科学院深圳先进技术研究院）
- 张镇嵩博士（华为诺亚2012实验室）
- 李明磊博士（零一万物）
- 马飞博士（光明实验室）
- 吴志勇老师（清华大学/香港中文大学）

另外，非常感谢常恒师兄 ( https://github.com/SwiftieH )、余伟江师兄的支持！

🎉 欢迎大家贡献自己的研究成果并PR，共同推动人体运动视频生成技术的发展。

如有任何问题，可以随时联系邮件（WinniyGD@outlook.com），我们会尽快回复。

另外，我们非常欢迎有新的相关领域的同学一同加入我们，一起学习，无限进步！

[2025/11/10] V15.4 Vision: Update Methods.

Shared Latent Representation for Joint Text-to-Audio-Visual Synthesis (Audio, Lip Synchronization)

[2025/11/04] V15.3 Vision: Update Methods.

ID-Composer: Multi-Subject Video Synthesis with Hierarchical Identity Preservation (Visual, Pose2Video)

Multi-View Consistent Human Image Customization via In-Context Learning (Visual, Pose2Video)

[2025/11/03] V15.2 Vision: Update Methods.

See the Speaker: Crafting High-Resolution Talking Faces from Speech with Prior Guidance and Region Refinement (Audio, Lip Synchronization)

[2025/10/28] V15.1 Vision: Update Methods.

MAGIC-Talk: Motion-aware Audio-Driven Talking Face Generation with Customizable Identity Control (Audio, Head Pose Driving)

Lookahead Anchoring: Preserving Character Identity in Audio-Driven Human Animation (Audio, Head Pose Driving)

[2025/10/21] V15.0 Vision: Update Methods.

From Mannequin to Human: A Pose-Aware and Identity-Preserving Video Generation Framework for Lifelike Clothing Display (Visual, Try-On Video Generation)

[2025/10/17] V14.9 Vision: Update Methods.

Identity-Preserving Image-to-Video Generation via Reward-Guided Optimization (Visual, Pose2Video)

[2025/10/15] V14.8 Vision: Update Methods.

Playmate2: Training-Free Multi-Character Audio-Driven Animation via Diffusion Transformer with Reward Feedback (Audio, Audio-Driven Holistic Body Driving)

[2025/10/14] V14.7 Vision: Update Methods.

DEMO: Disentangled Motion Latent Flow Matching for Fine-Grained Controllable Talking Portrait Synthesis (Audio, Head Pose Driving)

VividAnimator: An End-to-End Audio and Pose-driven Half-Body Human Animation Framework (Audio, Audio-Driven Holistic Body Driving)

[2025/10/12] V14.6 Vision: Update Methods.

Once Is Enough: Lightweight DiT-Based Video Virtual Try-On via One-Time Garment Appearance Injection (Visual, Try-On Video Generation)

[2025/10/09] V14.5 Vision: Update Methods.

A Bridge from Audio to Video: Phoneme-Viseme Alignment Allows Every Face to Speak Multiple Languages (Audio, Head Pose Driving)

[2025/10/06] V14.4 Vision: Update Methods.

Input-Aware Sparse Attention for Real-Time Co-Speech Video Generation (Audio, Audio-Driven Holistic Body Driving)

[2025/10/03] V14.3 Vision: Update Methods.

Audio Driven Real-Time Facial Animation for Social Telepresence (Audio, Head Pose Driving)

[2025/09/30] V14.2 Vision: Update Methods.

StableDub: Taming Diffusion Prior for Generalized and Efficient Visual Dubbing (Audio, Lip Synchronization)

X-Streamer: Unified Human World Modeling with Audiovisual Interaction (Audio, Head Pose Driving)

ControlHair: Physically-based Video Diffusion for Controllable Dynamic Hair Rendering (Visual, Pose2Video)

[2025/09/26] V14.1 Vision: Update Methods.

SynchroRaMa : Lip-Synchronized and Emotion-Aware Talking Face Generation via Multi-Modal Emotion Embedding (Audio, Head Pose Driving)

Talking Head Generation via AU-Guided Landmark Prediction (Audio, Head Pose Driving)

KSDiff: Keyframe-Augmented Speech-Aware Dual-Path Diffusion for Facial Animation (Audio, Head Pose Driving)

[2025/09/23] V14.0 Vision: Update Methods.

Stable Video-Driven Portraits (Visual, Portrait Animation)

Follow-Your-Emoji-Faster: Towards Efficient, Fine-Controllable, and Expressive Freestyle Portrait Animation (Visual, Portrait Animation)

[2025/09/18] V13.9 Vision: Update Methods.

Wan-Animate: Unified Character Animation and Replacement with Holistic Replication (Visual, Pose2Video)

[2025/09/16] V13.8 Vision: Update Methods.

AvatarSync: Rethinking Talking-Head Animation through Autoregressive Perspective (Audio, Head Pose Driving)

[2025/09/12] V13.7 Vision: Update Methods.

Kling-Avatar: Grounding Multimodal Instructions for Cascaded Long-Duration Avatar Animation Synthesis (Audio, Audio-Driven Holistic Body Driving)

[2025/09/05] V13.6 Vision: Update Methods.

Identity-Preserving Text-to-Video Generation via Training-Free Prompt, Image, and Guidance Enhancement (Text, Text2Motion)

[2025/08/27] V13.5 Vision: Update Methods.

MoCo: Motion-Consistent Human Video Generation via Structure-Appearance Decoupling (Visual, Pose2Video)

[2025/08/21] V13.4 Vision: Update Methods.

Taming Transformer for Emotion-Controllable Talking Face Generation (Audio, Head Pose Driving)

[2025/08/20] V13.3 Vision: Update Methods.

InfiniteTalk: Audio-driven Video Generation for Sparse-Frame Video Dubbing (Audio, Audio-Driven Holistic Body Driving)

EDTalk++: Full Disentanglement for Controllable Talking Head Synthesis (Audio, Head Pose Driving)

[2025/08/19] V13.2 Vision: Update Methods.

RealTalk: Realistic Emotion-Aware Lifelike Talking-Head Synthesis (Audio, Head Pose Driving)

[2025/08/18] V13.1 Vision: Update Methods.

FantasyTalking2: Timestep-Layer Adaptive Preference Optimization for Audio-Driven Portrait Animation (Audio, Audio-Driven Holistic Body Driving)

[2025/08/16] V13.0 Vision: Update Methods.

HM-Talker: Hybrid Motion Modeling for High-Fidelity Talking Head Synthesis (Audio, Head Pose Driving)

[2025/08/14] V12.9 Vision: Update Methods.

LIA-X: Interpretable Latent Portrait Animator (Visual, Portrait Animation)

HumanGenesis: Agent-Based Geometric and Generative Modeling for Synthetic Human Dynamics (Visual, Pose2Video)

From Large Angles to Consistent Faces: Identity-Preserving Video Generation via Mixture of Facial Experts (Text, Text2MotionVideo)

Animate-X++: Universal Character Image Animation with Dynamic Backgrounds (Visual, Pose2Video)

X-UniMotion: Animating Human Images with Expressive, Unified and Identity-Agnostic Motion Latents (Visual, Pose2Video)

[2025/08/13] V12.8 Vision: Update Methods.

ShoulderShot: Generating Over-the-Shoulder Dialogue Videos (Text, Text2MotionVideo)

Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generation (Text, Text2MotionVideo)

LaVieID: Local Autoregressive Diffusion Transformers for Identity-Preserving Video Creation (Text, Text2MotionVideo)

DiTalker: A Unified DiT-based Framework for High-Quality and Speaking Styles Controllable Portrait Animation (Audio, Head Pose Driving)

StableAvatar: Infinite-Length Audio-Driven Avatar Video Generation (Audio, Head Pose Driving)

AnimateScene: Camera-controllable Animation in Any Scene (Visual, Pose2Video)

RealisMotion: Decomposed Human Motion Control and Video Generation in the World Space (Visual, Pose2Video)

[2025/08/08] V12.7 Vision: Update Methods.

RAP: Real-time Audio-driven Portrait Animation with Video Diffusion Transformer (Audio, Head Pose Driving)

[2025/08/07] V12.6 Vision: Update Methods.

Motion is the Choreographer: Learning Latent Pose Dynamics for Seamless Sign Language Generation (Visual, Pose2Video)

[2025/08/06] V12.5 Vision: Update Methods.

READ: Real-time and Efficient Asynchronous Diffusion for Audio-driven Talking Head Generation (Audio, Head Pose Driving)

X-Actor: Emotional and Expressive Long-Range Portrait Acting from Audio (Audio, Head Pose Driving)

MoCA: Identity-Preserving Text-to-Video Generation via Mixture of Cross Attention (Text, Text2Face)

DreamVVT: Mastering Realistic Video Virtual Try-On in the Wild via a Stage-Wise Diffusion Transformer Framework (Visual, Try-On)

[2025/08/05] V12.4 Vision: Update Methods.

Text2Lip: Progressive Lip-Synced Talking Face Generation from Text via Viseme-Guided Rendering (Text, Text2Face)

[2025/08/01] V12.3 Vision: Update Methods.

X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attention (Visual, Portrait Animation)

[2025/07/29] V12.2 Vision: Update Methods.

Mask-Free Audio-driven Talking Face Generation for Enhanced Visual Quality and Identity Preservation (Audio, Head Pose Driving)

ChoreoMuse: Robust Music-to-Dance Video Generation with Style Transfer and Beat-Adherent Motion (Audio, Audio-Driven Holistic Body Driving)

[2025/07/28] V12.1 Vision: Update Methods.

Livatar-1: Real-Time Talking Heads Generation with Tailored Flow Matching (Audio, Head Pose Driving)

Face2VoiceSync: Lightweight Face-Voice Consistency for Text-Driven Talking Face Generation (Audio, Head Pose Driving)

[2025/07/24] V12.0 Vision: Update Methods.

CartoonAlive: Towards Expressive Live2D Modeling from Single Portraits (Visual, Portrait Animation)

[2025/07/23] V11.9 Vision: Update Methods.

HOComp: Interaction-Aware Human-Object Composition (Visual, Pose2Video)

Navigating Large-Pose Challenge for High-Fidelity Face Reenactment with Video Diffusion Model (Visual, Portrait Animation)

[2025/07/22] V11.8 Vision: Update Methods.

StableAnimator++: Overcoming Pose Misalignment and Face Distortion for Human Image Animation (Visual, Pose-Guided Dance Video Generation)

[2025/07/20] V11.7 Vision: Update Methods.

FantasyPortrait: Enhancing Multi-Character Portrait Animation with Expression-Augmented Diffusion Transformers (Text, Text2Face)

ATL-Diff: Audio-Driven Talking Head Generation with Early Landmarks-Guide Noise Diffusion (Audio, Head Pose Driving)

Think-Before-Draw: Decomposing Emotion Semantics & Fine-Grained Controllable Expressive Talking Head Generation (Audio, Head Pose Driving)

M2DAO-Talker: Harmonizing Multi-granular Motion Decoupling and Alternating Optimization for Talking-head Generation (Audio, Head Pose Driving)

HairShifter: Consistent and High-Fidelity Video Hair Transfer via Anchor-Guided Animation (Visual, Portrait Animation)

[2025/07/12] V11.6 Vision: Update Methods.

Stable-Hair v2: Real-World Hair Transfer via Multiple-View Diffusion Model (Visual, Portrait Animation)

[2025/07/10] V11.5 Vision: Update Methods.

Democratizing High-Fidelity Co-Speech Gesture Video Generation (Audio, Audio-Driven Holistic Body Driving)

[2025/07/09] V11.4 Vision: Update Methods.

MEDTalk: Multimodal Controlled 3D Facial Animation with Dynamic Emotions by Disentangled Embedding (Audio, Head Pose Driving)

[2025/07/08] V11.3 Vision: Update Methods.

MoDiT: Learning Highly Consistent 3D Motion Coefficients with Diffusion Transformer for Talking Head Generation (Audio, Head Pose Driving)

MoDA: Multi-modal Diffusion Architecture for Talking Head Generation (Audio, Head Pose Driving)

EchoMimicV3: 1.3B Parameters are All You Need for Unified Multi-Modal and Multi-Task Human Animation (Audio, Audio-Driven Holistic Body Driving)

Identity-Preserving Text-to-Video Generation Guided by Simple yet Effective Spatial-Temporal Decoupled Representations (Text, Text2MotionVideo)

[2025/07/04] V11.2 Vision: Update Methods.

CanonSwap: High-Fidelity and Consistent Video Face Swapping via Canonical Space Modulation (Visual, Portrait Animation)

[2025/07/03] V11.1 Vision: Update Methods.

FixTalk: Taming Identity Leakage for High-Quality Talking Head Generation in Extreme Cases (Audio, Head Pose Driving)

ARIG: Autoregressive Interactive Head Generation for Real-time Conversations (Audio, Head Pose Driving)

JAM-Flow: Joint Audio-Motion Synthesis with Flow Matching (Audio, Head Pose Driving)

GGTalker: Talking Head Systhesis with Generalizable Gaussian Priors and Identity-Specific Adaptation (Audio, Head Pose Driving)

Bind-Your-Avatar: Multi-Talking-Character Video Generation with Dynamic 3D-mask-based Embedding Router (Audio, Head Pose Driving)

Populate-A-Scene: Affordance-Aware Human Video Generation (Visual, Pose2Video)

Proteus-ID: ID-Consistent and Motion-Coherent Video Customization (Text, Text2MotionVideo)

MirrorMe: Towards Realtime and High Fidelity Audio-Driven Halfbody Animation (Audio, Audio-Driven Holistic Body Driving)

Video Virtual Try-on with Conditional Diffusion Transformer Inpainter (Visual, Try-On Video Generation)

[2025/06/24] V11.0 Vision: Update Methods.

OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animation (Audio, Audio-Driven Holistic Body Driving)

[2025/06/23] V10.9 Vision: Update Methods.

Controllable and Expressive One-Shot Video Head Swapping (Visual, Portrait Animation)

EchoShot: Multi-Shot Portrait Video Generation (Text, Text2MotionVideo)

[2025/06/18] V10.8 Vision: Update Methods.

3D Hand Mesh-Guided AI-Generated Malformed Hand Refinement with Hand Pose Transformation via Diffusion Model (Visual, Pose-Guided Dance Video Generation)

AlignHuman: Improving Motion and Fidelity via Timestep-Segment Preference Optimization for Audio-Driven Human Animation (Audio, Audio-Driven Holistic Body Driving)

Audio-Visual Driven Compression for Low-Bitrate Talking Head Videos (Audio, Head Pose Driving)

[2025/06/13] V10.7 Vision: Update Methods.

Low-Barrier Dataset Collection with Real Human Body for Interactive Per-Garment Virtual Try-On (Visual, Try-On Video Generation)

DreamActor-H1: High-Fidelity Human-Product Demonstration Video Generation via Motion-designed Diffusion Transformers (Visual, Pose2Video)

[2025/06/12] V10.6 Vision: Update Methods.

Synthetic Human Action Video Data Generation with Pose Transfer (Visual, Pose2Video)

InterActHuman: Multi-Concept Human Animation with Layout-Aligned Audio Conditions (Audio, Audio-Driven Holistic Body Driving)

[2025/06/09] V10.5 Vision: Update Methods.

ChronoTailor: Harnessing Attention Guidance for Fine-Grained Video Virtual Try-On (Visual, Try-On Video Generation)

LLIA -- Enabling Low-Latency Interactive Avatars: Real-Time Audio-Driven Portrait Video Generation with Diffusion Models (Audio, Head Pose Driving)

[2025/06/04] V10.4 Vision: Update Methods.

OmniV2V: Versatile Video Generation and Editing via Dynamic Content Manipulation (Visual, Pose-Guided Dance Video Generation)

SkyReels-Audio: Omni Audio-Conditioned Talking Portraits in Video Diffusion Transformers (Audio, Head Pose Driving)

[2025/06/03] V10.3 Vision: Update Methods.

DreamDance: Animating Character Art via Inpainting Stable Gaussian Worlds (Visual, Pose-Guided Dance Video Generation)

[2025/05/30] V10.2 Vision: Update Methods.

HyperMotion: DiT-Based Pose-Guided Human Image Animation of Complex Motions (Visual, Pose-Guided Dance Video Generation)

How Animals Dance (When You're Not Looking) (Visual, Pose-Guided Dance Video Generation)

Video Editing for Audio-Visual Dubbing (Audio, Lip Synchronization)

Hallo4: High-Fidelity Dynamic Portrait Animation via Direct Preference Optimization and Temporal Motion Modulation (Audio, Head Pose Driving)

MMGT: Motion Mask Guided Two-Stage Network for Co-Speech Gesture Video Generation (Audio, Audio-Driven Holistic Body Driving)

[2025/05/29] V10.1 Vision: Update Methods.

LatentMove: Towards Complex Human Movement Video Generation (Visual, Pose-Guided Dance Video Generation)

Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation (Audio, Head Pose Driving)

FaceEditTalker: Interactive Talking Head Generation with Facial Attribute Editing (Audio, Head Pose Driving)

OmniResponse: Online Multimodal Conversational Response Generation in Dyadic Interactions (LLM, Large Lanuge Model Planer)

[2025/05/28] V10.0 Vision: Update Methods.

MagicTryOn: Harnessing Diffusion Transformer for Garment-Preserving Video Virtual Try-on (Visual, Try-On Video Generation)

OmniSync: Towards Universal Lip Synchronization via Diffusion Transformers (Audio, Head Pose Driving)

[2025/05/27] V9.9 Vision: Update Methods.

DanceTogether! Identity-Preserving Multi-Person Interactive Video Generation (Visual, Video-Guided Dance Video Generation)

HunyuanVideo-Avatar: High-Fidelity Audio-Driven Human Animation for Multiple Characters (Visual, Audio-Driven Holistic Body Driving)

AniCrafter: Customizing Realistic Human-Centric Animation via Avatar-Background Conditioning in Video Diffusion Models (Visual, Pose2Video)

[2025/05/23] V9.8 Vision: Update Methods.

Interspatial Attention for Efficient 4D Human Video Generation (Visual, Pose-Guided Dance Video Generation)

[2025/05/13] V9.7 Vision: Update Methods.

ProFashion: Prototype-guided Fashion Video Generation with Multiple Reference Images (Visual, Try-On Video Generation)

[2025/05/09] V9.6 Vision: Update Methods.

FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarios (Visual, Video-Guided Dance Video Generation)

PAHA: Parts-Aware Audio-Driven Human Animation with Diffusion Model (Audio, Audio-Driven Holistic Body Driving)

HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generation (Audio, Audio-Driven Holistic Body Driving)

[2025/05/01] V9.5 Vision: Update Methods.

MagicPortrait: Temporally Consistent Face Reenactment with 3D Geometric Guidance (Visual, Portrait Animation)

[2025/04/30] V9.4 Vision: Update Methods.

RealisDance-DiT: Simple yet Strong Baseline towards Controllable Character Animation in the Wild (Visual, Pose-Guided Dance Video Generation)

AnimateAnywhere: Rouse the Background in Human Image Animation (Visual, Pose-Guided Dance Video Generation)

IM-Portrait: Learning 3D-aware Video Diffusion for Photorealistic Talking Heads from Monocular Videos (Visual, Portrait Animation)

3DV-TON: Textured 3D-Guided Consistent Video Try-on via Diffusion Models (Visual, Try-On Video Generation)

Audio-Driven Talking Face Video Generation with Joint Uncertainty Learning (Audio, Head Pose Driving)

Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional Talking Portrait Generation (Audio, Head Pose Driving)

[2025/04/16] V9.3 Vision: Update Methods.

UniAnimate-DiT: Human Image Animation with Large-Scale Video Diffusion Transformer (Visual, Pose-Guided Dance Video Generation)

InterAnimate: Taming Region-aware Diffusion Model for Realistic Human Interaction Animation (Visual, Portrait Animation)

[2025/04/14] V9.2 Vision: Update Methods.

TokenMotion: Decoupled Motion Control via Token Disentanglement for Human-centric Video Generation (Visual, Pose-Guided Dance Video Generation)

EasyGenNet: An Efficient Framework for Audio-Driven Gesture Video Generation Based on Diffusion Model (Audio, Audio-Driven Holistic Body Driving)

[2025/04/09] V9.1 Vision: Update Methods.

Multi-identity Human Image Animation with Structural Video Diffusion (Visual, Pose-Guided Dance Video Generation)

FantasyTalking: Realistic Talking Portrait Generation via Coherent Motion Synthesis (Audio, Head Pose Driving)

SE4Lip: Speech-Lip Encoder for Talking Head Synthesis to Solve Phoneme-Viseme Alignment Ambiguity (Audio, Head Pose Driving)

Exploiting Temporal Audio-Visual Correlation Embedding for Audio-Driven One-Shot Talking Head Animation (Audio, Head Pose Driving)

FluentLip: A Phonemes-Based Two-stage Approach for Audio-Driven Lip Synthesis with Optical Flow Consistency (Audio, Lip Synchronization)

[2025/04/04] V9.0 Vision: Update Methods.

DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidance (Visual, Video-Guided Dance Video Generation)

OmniTalker: Real-Time Text-Driven Talking Head Generation with In-Context Audio-Visual Style Replication (Text, Text2Face)

Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generation (Audio, Head Pose Driving)

VoiceCraft-Dub: Automated Video Dubbing with Neural Codec Language Models (Audio, Head Pose Driving)

[2025/04/02] V8.9 Vision: Update Methods.

Monocular and Generalizable Gaussian Talking Head Animation (Audio, Head Pose Driving)

[2025/04/01] V8.8 Vision: Update Methods.

HumanDreamer: Generating Controllable Human-Motion Videos via Decoupled Generation (Text, Text2MotionVideo)

Dual Audio-Centric Modality Coupling for Talking Head Generation (Audio, Head Pose Driving)

[2025/03/31] V8.7 Vision: Update Methods.

Follow Your Motion: A Generic Temporal Consistency Portrait Editing Framework with Trajectory Guidance (Visual, Pose2Video)

High-Fidelity Diffusion Face Swapping with ID-Constrained Facial Conditioning (Visual, Portrait Animation)

Audio-Plane: Audio Factorization Plane Gaussian Splatting for Real-Time Talking Head Synthesis (Audio, Head Pose Driving)

[2025/03/26] V8.6 Vision: Update Methods.

EmoHead: Emotional Talking Head via Manipulating Semantic Expression Parameters (Audio, Fine-Grained Style and Emotion-Driven Animation)

MVPortrait: Text-Guided Motion and Emotion Control for Multi-view Vivid Portrait Animation (Audio, Fine-Grained Style and Emotion-Driven Animation)

DisentTalk: Cross-lingual Talking Face Generation via Semantic Disentangled Diffusion Model (Audio, Fine-Grained Style and Emotion-Driven Animation)

AudCast: Audio-Driven Human Video Generation by Cascaded Diffusion Transformers (Audio, Audio-Driven Holistic Body Driving)

[2025/03/25] V8.5 Vision: Update Methods.

HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animation (Visual, Portrait Animation)

EvAnimate: Event-conditioned Image-to-Video Generation for Human Animation (Visual, Pose-Guided Dance Video Generation)

Teller: Real-Time Streaming Audio-Driven Portrait Animation with Autoregressive Motion Generation (Audio, Head Pose Driving)

[2025/03/24] V8.4 Vision: Update Methods.

Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Model (Visual, Pose2Video)

[2025/03/21] V8.3 Vision: Update Methods.

Text-Driven Diffusion Model for Sign Language Production (Text, Text2MotionVideo)

[2025/03/20] V8.2 Vision: Update Methods.

Cafe-Talk: Generating 3D Talking Face Animation with Multimodal Coarse- and Fine-grained Control (Audio, Head Pose Driving)

[2025/03/19] V8.1 Vision: Update Methods.

Concat-ID: Towards Universal Identity-Preserving Video Synthesis (Text, Text2Face)

PC-Talk: Precise Facial Animation Control for Audio-Driven Talking Face Generation (Audio, Head Pose Driving)

MusicInfuser: Making Video Diffusion Listen and Dance (Audio, Audio-Driven Holistic Body Driving)

[2025/03/18] V8.0 Vision: Update Methods.

RASA: Replace Anyone, Say Anything -- A Training-Free Framework for Audio-Driven and Universal Portrait Video Editing (Audio, Head Pose Driving)

MagicID: Hybrid Preference Optimization for ID-Consistent and Dynamic-Preserved Video Customization (Text, Text2MotionVideo)

Snapmoji: Instant Generation of Animatable Dual-Stylized Avatars (Visual, Portrait Animation)

[2025/03/15] V7.9 Vision: Update Methods.

Semantic Latent Motion for Portrait Video Generation (Visual, Portrait Animation)

Cosh-DiT: Co-Speech Gesture Video Synthesis via Hybrid Audio-Visual Diffusion Transformers (Audio, Audio-Driven Holistic Body Driving)

[2025/03/13] V7.8 Vision: Update Methods.

SwapAnyone: Consistent and Realistic Video Synthesis for Swapping Any Person into Any Video (Visual, Pose2Video)

Versatile Multimodal Controls for Whole-Body Talking Human Animation (Audio, Audio-Driven Holistic Body Driving)

[2025/03/11] V7.7 Vision: Update Methods.

DynamicID: Zero-Shot Multi-ID Image Personalization with Flexible Facial Editability (Text, Text2Face)

[2025/03/07] V7.6 Vision: Update Methods.

FREAK: Frequency-modulated High-fidelity and Real-time Audio-driven Talking Portrait Synthesis (Audio, Head Pose Driving)

[2025/03/05] V7.5 Vision: Update Methods.

FaceShot: Bring Any Character into Life (Visual, Portrait Animation)

KeyFace: Expressive Audio-Driven Facial Animation for Long Sequences via KeyFrame Interpolation (Audio, Head Pose Driving)

[2025/02/28] V7.4 Vision: Update Methods.

High-Fidelity Relightable Monocular Portrait Animation with Lighting-Controllable Video Diffusion Model (Visual, Portrait Animation)

InsTaG: Learning Personalized 3D Talking Head from Few-Second Video (Audio, Head Pose Driving)

[2025/02/26] V7.3 Vision: Update Methods.

FantasyID: Face Knowledge Enhanced ID-Preserving Video Generation (Text, Text2MotionVideo)

[2025/02/25] V7.2 Vision: Update Methods.

X-Dancer: Expressive Music to Human Dance Video Generation (Audio, Audio-Driven Holistic Body Driving)

Dimitra: Audio-driven Diffusion model for Expressive Talking Head Generation (Audio, Head Pose Driving)

[2025/02/21] V7.1 Vision: Update Methods.

NeRF-3DTalker: Neural Radiance Field with 3D Prior Aided Audio Disentanglement for Talking Head Synthesis (Audio, Head Pose Driving)

[2025/02/18] V7.0 Vision: Update Methods.

HumanGif: Single-View Human Diffusion with Generative Prior (Visual, Pose-Guided Dance Video Generation)

SayAnything: Audio-Driven Lip Synchronization with Conditional Video Diffusion (Audio, Head Pose Driving)

SkyReels-A1: Expressive Portrait Animation in Video Diffusion Transformers (Audio, Head Pose Driving)

[2025/02/16] V6.9 Vision: Update Methods.

Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model (Audio, Head Pose Driving)

[2025/02/13] V6.8 Vision: Update Methods.

AnyCharV: Bootstrap Controllable Character Video Generation with Fine-to-Coarse Guidance (Visual, Pose-Guided Dance Video Generation)

[2025/02/12] V6.7 Vision: Update Methods. Happy 2025 !!🍟

Contextual Gesture: Co-Speech Gesture Video Generation through Context-aware Gesture Representation (Audio, Audio-Driven Holistic Body Driving)

Playmate: Flexible Control of Portrait Animation via 3D-Implicit Space Guided Diffusion (Audio, Fine-Grained Style and Emotion-Driven Animation)

[2025/02/11] V6.6 Vision: Update Methods. Happy 2025 !!🍟

Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance (Visal, Pose-Guided Dance Video Generation)

[2025/02/10] V6.5 Vision: Update Methods. Happy 2025 !!🍟

HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generation (Visal, Pose2Video)

[2025/02/04] V6.4 Vision: Update Methods. Happy 2025 !!🍟

EmoTalkingGaussian: Continuous Emotion-conditioned Talking Head Synthesis (Audio, Head Pose Driving)

OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models (Audio, Audio-Driven Holistic Body Driving)

[2025/02/03] V6.3 Vision: Update Methods. Happy 2025 !!🍟

Every Image Listens, Every Image Dances: Music-Driven Image Animation (Audio, Audio-Driven Holistic Body Driving)

[2025/01/30] V6.2 Vision: Update Methods. Happy 2025 !!🍟

Learning Semantic Facial Descriptors for Accurate Face Animation (Visual, Portrait Animation)

[2025/01/28] V6.1 Vision: Update Methods. Happy 2025 !!🍟

SyncAnimation: A Real-Time End-to-End Framework for Audio-Driven Human Pose and Talking Head Animation (Audio, Head Pose Driving)

[2025/01/24] V6.0 Vision: Update Methods. Happy New Year🍟

EchoVideo: Identity-Preserving Human Video Generation by Multimodal Feature Fusion (Text, Text2MotionVideo)

[2025/01/22] V5.9 Vision: Update Methods. Happy New Year🎀

CatV2TON: Taming Diffusion Transformers for Vision-Based Virtual Try-On with Temporal Concatenation (Visual, Try-On Video Generation)

EMO2: End-Effector Guided Audio-Driven Avatar Video Generation (Audio, Audio-Driven Holistic Body Driving)

[2025/01/20] V5.8 Vision: Update Methods. Happy New Year🎀

X-Dyna: Expressive Dynamic Human Image Animation (Visual, Pose-Guided Dance Video Generation)

Textoon: Generating Vivid 2D Cartoon Characters from Text Descriptions (Text, Text2Motion)

[2025/01/18] V5.7 Vision: Update Methods. Happy New Year🎀

RealVVT: Towards Photorealistic Video Virtual Try-on via Spatio-Temporal Consistency (Visual, Try-On Video Generation)

DynamicFace: High-Quality and Consistent Video Face Swapping using Composable 3D Facial Priors (Visual, Portrait Animation)

[2025/01/16] V5.6 Vision: Update Methods. Happy New Year🎀

Qffusion: Controllable Portrait Video Editing via Quadrant-Grid Attention Learning (Visual, Portrait Animation)

[2025/01/14] V5.5 Vision: Update Methods. Happy New Year🎀

Identity-Preserving Video Dubbing Using Motion Warping (Audio, Lip Synchronization)

[2025/01/13] V5.4 Vision: Update Methods. Happy New Year🎀

Ingredients: Blending Custom Photos with Video Diffusion Transformers (Text, Text2MotionVideo)

Magic Mirror: ID-Preserved Video Generation in Video Diffusion Transformers (Text, Text2MotionVideo)

[2025/01/12] V5.3 Vision: Update Methods. Happy New Year🎀

MoEE: Mixture of Emotion Experts for Audio-Driven Portrait Animation (Audio, Head Pose Driving)

[2025/01/11] V5.2 Vision: Update Methods. Happy New Year🎀

UniAvatar: Taming Lifelike Audio-Driven Talking Head Generation with Comprehensive Motion and Lighting Control (Audio, Head Pose Driving)

[2025/01/10] V5.1 Vision: Update Methods. Happy New Year🎀

RAIN: Real-time Animation of Infinite Video Stream (Visual, Portrait Animation)

VideoMaker: Zero-shot Customized Video Generation with the Inherent Force of Video Diffusion Models (Text, Text2Face)

[2025/01/06] V5.0 Vision: Update Methods. Happy New Year🎀

Free-viewpoint Human Animation with Pose-correlated Reference Selection (Visual, Pose-Guided Dance Video Generation)

ManiVideo: Generating Hand-Object Manipulation Video with Dexterous and Generalizable Grasping (Visual, Pose2Video)

Follow-Your-MultiPose: Tuning-Free Multi-Character Text-to-Video Generation via Pose Guidance (Text, Text2MotionVideo)

[2025/01/04] V4.9 Vision: Update Methods. Happy New Year🎀

Consistent Human Image and Video Generation with Spatially Conditioned Diffusion (Visual, Pose-Guided Dance Video Generation)

[2024/12/17] V4.8 Vision: Update Methods.

VividFace: A Diffusion-Based Hybrid Framework for High-Fidelity Video Face Swapping (Visual, Portrait Animation)

VQTalker: Towards Multilingual Talking Avatars through Facial Motion Tokenization (Audio, Fine-Grained Style and Emotion-Driven Animation)

Dynamic Try-On: Taming Video Virtual Try-on with Dynamic Attention Mechanism (Visual, Try-On Video Generation)

SwiftTry: Fast and Consistent Video Virtual Try-On with Diffusion Models (Visual, Try-On Video Generation)

[2024/12/15] V4.7 Vision: Update Methods.

LatentSync: Audio Conditioned Latent Diffusion Models for Lip Sync (Audio, Lip Synchronization)

GoHD: Gaze-oriented and Highly Disentangled Portrait Animation with Rhythmic Poses and Realistic Expression (Audio, Head Pose Driving)

PortraitTalk: Towards Customizable One-Shot Audio-to-Talking Face Generation (Audio, Head Pose Driving)

IF-MDM: Implicit Face Motion Diffusion Model for High-Fidelity Realtime Talking Head Generation (Audio, Head Pose Driving)

INFP: Audio-Driven Interactive Head Generation in Dyadic Conversations (Audio, Head Pose Driving)

MEMO: Memory-Guided Diffusion for Expressive Talking Video Generation (Audio, Head Pose Driving)

DisPose: Disentangling Pose Guidance for Controllable Human Image Animation (Visual, Pose-Guided Dance Video Generation)

[2024/12/11] V4.6 Vision: Update Methods.

PEMF-VVTO: Point-Enhanced Video Virtual Try-on via Mask-free Paradigm (Visual, Try-On Video Generation)

SINGER: Vivid Audio-driven Singing Video Generation with Multi-scale Spectral Diffusion Model (Audio, Head Pose Driving)

EmojiDiff: Advanced Facial Expression Control with High Identity Preservation in Portrait Generation (Visual, Portrait Animation)

Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Diffusion Transformer Networks (Visual, Portrait Animation)

FLOAT: Generative Motion Latent Flow Matching for Audio-driven Talking Portrait (Visual, Portrait Animation)

DreamDance: Animating Human Images by Enriching 3D Geometry Cues from 2D Poses (Visual, Pose-Guided Dance Video Generation)

[2024/12/02] V4.5 Vision: Update Methods.

LokiTalk: Learning Fine-Grained and Generalizable Correspondences to Enhance NeRF-based Talking Head Synthesis (Audio, Head Pose Driving)

Ditto: Motion-Space Diffusion for Controllable Realtime Talking Head Synthesis (Audio, Head Pose Driving)

Fleximo: Towards Flexible Text-to-Human Motion Video Generation (Text, Text2MotionVideo)

[2024/11/28] V4.4 Vision: Update Methods.

HiFiVFS: High Fidelity Video Face Swapping (Visual, Portrait Animation)

MotionCharacter: Identity-Preserving and Motion Controllable Human Video Generation (Text, Text2Face)

Identity-Preserving Text-to-Video Generation by Frequency Decomposition (Text, Text2Face)

AnchorCrafter: Animate CyberAnchors Saling Your Products via Human-Object Interacting Video Generation (Visual, Pose2Video)

PersonalVideo: High ID-Fidelity Video Customization without Dynamic and Semantic Degradation (Text, Text2Face)

LetsTalk: Latent Diffusion Transformer for Talking Video Synthesis (Audio, Fine-Grained Style and Emotion-Driven Animation)

EmotiveTalk: Expressive Talking Head Generation through Audio Information Decoupling and Emotional Video Diffusion (Audio, Fine-Grained Style and Emotion-Driven Animation)

Sonic: Shifting Focus to Global Audio Perception in Portrait Animation (Audio, Fine-Grained Style and Emotion-Driven Animation)

StableAnimator: High-Quality Identity-Preserving Human Image Animation (Visual, Pose-Guided Dance Video Generation)

[2024/11/25] V4.3 Vision: Update Methods.

FloAt: Flow Warping of Self-Attention for Clothing Animation Generation (Visual, Try-On Video Generation)

[2024/11/18] V4.2 Vision: Update Methods.

EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation (Audio, Audio-Driven Holistic Body Driving)

[2024/11/15 !WoW! More Star 100 🌟🌟🌟] V4.1 Vision: Update Methods.

JoyVASA: Portrait and Animal Image Animation with Diffusion-Based Audio-Driven Facial Dynamics and Head Motion Generation (Audio, Fine-Grained Style and Emotion-Driven Animation)

LES-Talker: Fine-Grained Emotion Editing for Talking Head Generation in Linear Emotion Space (Audio, Fine-Grained Style and Emotion-Driven Animation)

[2024/11/14]V4.0 Vision: Update Methods.

MikuDance: Animating Character Art with Mixed Motion Dynamics (Visual, Pose-Guided Dance Video Generation)

[2024/11/04]V3.9 Vision: Update Methods.

Fashion-VDM (Visual, Try-On Video Generation)

Towards High-fidelity Head Blending with Chroma Keying for Industrial Applications (Visual, Portrait Animation)

[2024/11/01]V3.8 Vision: Update Methods.

Stereo-Talker (Audio, Audio-Driven Holistic Body Driving)

[2024/10/29]V3.7 Vision: Update Methods.

MovieCharacter (Visual, Pose2Video)

[2024/10/24 Happy Coding Day!]V3.6 Vision: Update Methods.

EmoGene (Audio, Fine-Grained Style and Emotion-Driven Animation)

[2024/10/21]V3.5 Vision: Update Methods.

Takin-ADA: Emotion Controllable Audio-Driven Animation with Canonical and Landmark Loss Optimization (Audio, Fine-Grained Style and Emotion-Driven Animation)

[2024/10/18]V3.4 Vision: Update Methods.

DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework for Talking Head Video Generation (Audio, Head Pose Driving)

[2024/10/15]V3.3 Vision: Update Methods.

Tex4D (Text, Text2MotionVideo)

TALK-Act: Enhance Textural-Awareness for 2D Speaking Avatar Reenactment with Diffusion Model (Audio, Audio-Driven Holistic Body Driving)

Animate-X: Universal Character Image Animation with Enhanced Motion Representation (Visual, Pose-Guided Dance Video Generation)

MuseTalk: Real-Time High Quality Lip Synchronization with Latent Space Inpainting (Audio, Lip Synchronization)

[2024/10/11]V3.2 Vision: Update Methods.

Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animation (Audio, Fine-Grained Style and Emotion-Driven Animation)

[2024/10/10]V3.1 Vision: Update Methods.

MimicTalk: Mimicking a personalized and expressive 3D talking face in minutes (Audio, Fine-Grained Style and Emotion-Driven Animation)

[2024/10/08]V3.0 Vision: Update Methods.

TANGO (Audio, Audio-Driven Holistic Body Driving)

[2024/10/04]🎉🎉🎉V2.9 Vision I'm glad that our article is publicly available on TechRxiv. We welcome your attention and citations. The version on arXiv is still on hold, and we will update it when it becomes available.

@article{xue2024human,
  title={Human Motion Video Generation: A Survey}, 
  author={Xue, Haiwei and Luo, Xiangyang and Hu, Zhanghao and Zhang, Xin and Xiang, Xunzhi and Dai, Yuqin and Liu, Jianzhuang and Zhang, Zhensong and Li, Minglei and Yang, Jian and Ma, Fei and Wu, Zhiyong and Yang, Changpeng and Dai, Zonghong and Yu, Fei Richard},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, 
  year={2025},
  doi={10.1109/TPAMI.2025.3594034}
}

[2024/10/03]V2.8 Vision: Update Methods.

LaDTalk: Latent Denoising for Synthesizing Talking Head Videos with High Frequency Details (Audio, Head Pose Driving)

[2024/10/02]V2.7 Vision: Update Methods.

Replace Anyone in Videos (Visual, Video-Guided Dance Video Generation)

High Quality Human Image Animation using Regional Supervision and Motion Blur Condition (Visual, Pose-Guided Dance Video Generation)

[2024/09/27]V2.6 Vision: Update Methods.

SVP (Visual, Portrait Animation)

Self-Supervised Learning of Deviation in Latent Representation for Co-speech Gesture Video Generation (Audio, Audio-Driven Holistic Body Driving)

[2024/09/25]V2.5 Vision: Update Methods.

MIMO (Visual, Pose-Guided Dance Video Generation)

[2024/09/24]V2.4 Vision: Update Methods.

MIMAFace (Audio, Fine-Grained Style and Emotion-Driven Animation)

[2024/09/23]V2.3 Vision: Update Methods.

JoyHallo (Audio, Fine-Grained Style and Emotion-Driven Animation)

[2024/09/19] V2.2 Vision: Update Methods.

JEAN (Audio, Head Pose Driving)

[2024/09/17] V2.1 Vision: Update Methods.

LawDNet (Audio, Lip Synchronization)

StyleTalk++ (Audio, Fine-Grained Style and Emotion-Driven Animation)

[2024/09/13] V2.0 Vision: Update Methods.

DiffTED (Audio, Audio-Driven Holistic Body Driving)

[2024/09/12] V1.9 Vision: Update Methods.

EMOdiffhead (Audio, Fine-Grained Animation)

[2024/09/11] V1.8 Vision: Update Methods.

RealisDance (Visual, Pose-Guided Dance Video Generation)

[2024/09/10] V1.7 Vision: Update Methods.

Leveraging WaveNet for Dynamic Listening Head Modeling from Speech (Audio, Lip Synchronization)

KAN-Based Fusion of Dual-Domain for Audio-Driven Facial Landmarks Generation (Audio, Lip Synchronization)

PersonaTalk (Audio, Lip Synchronization)

[2024/09/06] V1.6 Vision: Update Methods.

SVP (Audio, Fine-Grained Animation)

SegTalker (Audio, Lip Synchronization)

[2024/09/05] V1.5 Vision: Update Methods.

Loopy (Audio, Fine-Grained Animation)

PoseTalk (Audio, Fine-Grained Animation)

[2024/09/04] V1.4 Vision: Update Methods.

CyberHost (Audio, Holistic Human Driving)

[2024/08/28] V1.3 Vision: Update Methods.

MegActor-Σ (Audio, Fine-Grained Animation)

Rafael Azevedo et al. (Text, Text2Face)

[2024/08/27] V1.2 Vision: Update Methods.

GenCA (Text, Text2Face)

[2024/08/26] V1.1 Vision: Update Methods.

G3FA (Vision, Portrait Animation)