World Action Models: The Next Frontier in Embodied AI
July 21, 2026 · View on GitHub
World Action Models: The Next Frontier in Embodied AI
Siyin Wang1,2,*,‡,
Junhao Shi1,2,*,
Zhaoyang Fu1,*,
Xinzhe He1,*,
Feihong Liu1,*,
Chenchen Yang1,2,
Yikang Zhou2,
Zhaoye Fei1,
Jingjing Gong2,
Jinlan Fu1
,
Mike Zheng Shou3 ·
Xuanjing Huang1,2 ·
Xipeng Qiu1,2 ·
Yu-Gang Jiang1,†
1Fudan University,
2Shanghai Innovation Institute,
3National University of Singapore
*Equal Contribution,
‡Project Lead,
†Corresponding Author
This repository accompanies our survey on World Action Models (WAMs) — the emerging paradigm that unifies predictive world modeling with action generation for embodied AI. We will keep this repo continuously updated as the field evolves.
- 📄 The first systematic WAM survey, covering architecture taxonomy (Cascaded & Joint WAMs), training data, evaluation protocols, and world models for VLA learning.
- 📖 Reading blogs included: we also provide a concise, structured summary blog for each paper to help you quickly grasp the key ideas, architecture, and contributions. The summarization skill used to generate them is also open-sourced in this repository (Check Paper2Blog).
- 🤝 Community-driven: found a missing paper or have a suggestion? Feel free to open an issue or submit a pull request!
Welcome to join our Feishu discussion group for in-depth academic exchange on World-Action Models, robot learning, and embodied intelligence. Our bot will also periodically share the latest WAM-related papers and updates, helping everyone stay informed about recent progress and discuss new ideas together.
🔔 News
-
[2026-07-01] We are excited to share that the repository has surpassed 1,000 stars. Thank you all for your support!
-
[2026-06-24] Added the community discussion group QR code, and everyone is welcome to join and exchange ideas. (Use the QR code above or check Feishu/Lark)
-
[2026-05-25] Added the benchmark leaderboard page and benchmark-level performance trend visualization.
-
[2026-05-21] Updated the link to the paper-reading skill.
-
[2026-05-13] Initial release of the survey paper and repository.
Contents
Tag Legend
World Action Model tags
-
Cascaded WAM
-
Pixel-space Representations
Learned Action Extraction
Geometric Extraction
-
Implicit Planning via Latent Representations
-
Joint WAM
-
Autoregressive Generation
Explicit Decoupled Representation
Unified Discrete Representations
Predictive Latent Representation
-
Diffusion-based Generation
Unified Stream
Explicit Future Generation
Implicit Future Prediction
Multi-Stream
Cross-Attention Coupling
Hidden-State Coupling
Shared Representation
-
Cache: "Cache: Accelerating World Action Models with Cross Inference Chunk Cache", arXiv 2026.
[📄 Paper]
World Action Model
Cascaded World-Action-Model
-
UniPi: "Learning Universal Policies via Text-Guided Video Generation", NeurIPS 2023.
-
VLP: "Video Language Planning", ICLR 2024.
-
RoboEnvision: "RoboEnvision: A Long-Horizon Video Generation Model for Multi-Task Robot Manipulation", IROS 2025.
-
ThisAndThat: "This&That: Language-Gesture Controlled Video Generation for Robot Planning", ICRA 2025.
-
TesserAct: "TesserAct: Learning 4D Embodied World Models", ICCV 2025.
-
Gen2Act: "Gen2Act: Human Video Generation in Novel Scenarios Enables Generalizable Robot Manipulation", CoRL 2025.
-
Vidar: "Vidar: Embodied Video Diffusion Model for Generalist Bimanual Manipulation", arXiv 2025.
-
MVISTA-4D: "MVISTA-4D: View-Consistent 4D World Model with Test-Time Action Inference for Robotic Manipulation", arXiv 2026.
-
SayDreamAct: "Say, Dream, and Act: Learning Video World Models for Instruction-Driven Robot Manipulation", arXiv 2026.
-
Veo-Act: "Veo-Act: How Far Can Frontier Video Models Advance Generalizable Robot Manipulation?", arXiv 2026.
-
VAG: "VAG: Dual-Stream Video-Action Generation for Embodied Data Synthesis", arXiv 2026.
-
pi0.7: "pi0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities", arXiv 2026.
-
AVDC: "Learning to Act from Actionless Videos through Dense Correspondences", ICLR 2024.
-
Im2Flow2Act: "Flow as the Cross-Domain Manipulation Interface", CoRL 2024.
-
Dreamitate: "Dreamitate: Real-World Visuomotor Policy Learning via Video Generation", CoRL 2024.
-
3DFlowAction: "3DFlowAction: Learning Cross-Embodiment Manipulation from 3D Flow World Model", arXiv 2025.
-
NovaFlow: "NovaFlow: Zero-Shot Manipulation via Actionable Flow from Generated Videos", arXiv 2025.
-
LV-P: "Large Video Planner Enables Generalizable Robot Control", arXiv 2025.
-
Dream2Flow: "Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flow", arXiv 2025.
-
4DGen: "Geometry-aware 4D Video Generation for Robot Manipulation", ICLR 2026.
-
RIGVid: "Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrations", ICLR 2026.
-
ARDuP: "ARDuP: Active Region Video Diffusion for Universal Policies", IROS 2024.
-
VPP: "Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representations", ICML 2025.
-
VILP: "VILP: Imitation Learning with Latent Video Planning", RA-L 2025.
-
LaPA: "Latent Action Pretraining from Videos", ICLR 2025.
-
VideoPolicy: "Video Generators are Robot Policies", arXiv 2025.
-
mimic-video: "mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs", arXiv 2025.
-
villa-X: "villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Models", ICLR 2026.
-
S-VAM: "S-VAM: Shortcut Video-Action Model by Self-Distilling Geometric and Semantic Foresight", arXiv 2026.
-
OmniVTA: "OmniVTA: Visuo-Tactile World Modeling for Contact-Rich Robotic Manipulation", arXiv 2026.
-
MWM: "Mask World Model: Predicting What Matters for Robust Robot Policy Learning", arXiv 2026.
-
FAWAM: "FAWAM: Force-Aware World Action Models for Closed-Loop Contact-Rich Manipulation", arXiv 2026.
-
World Pilot: "World Pilot: Steering Vision-Language-Action Models with World-Action Priors", arXiv 2026.
Joint World-Action-Model
Autoregressive Generation
-
GR-1: "Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation", ICLR 2024.
-
GR-2: "GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation", arXiv 2024.
-
GR-MG: "GR-MG: Leveraging Partially-Annotated Data via Multi-Modal Goal-Conditioned Policy", RA-L 2025.
-
CoT-VLA: "CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models", CVPR 2025.
-
WorldVLA: "WorldVLA: Towards Autoregressive Action World Model", arXiv 2025.
-
F1: "F1: A Vision-Language-Action Model Bridging Understanding and Generation to Actions", arXiv 2025.
-
RynnVLA-002: "RynnVLA-002: A Unified Vision-Language-Action and World Model", arXiv 2025.
-
VLA-JEPA: "VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model", arXiv 2026.
Diffusion-based Generation
-
PAD: "Prediction with Action: Visual Policy Learning via Joint Denoising Process", NeurIPS 2024.
-
VideoVLA: "VideoVLA: Video Generators Can Be Generalizable Robot Manipulators", NeurIPS 2025.
-
UWM: "Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets", RSS 2025.
-
Cosmos Policy: "Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning", arXiv 2026.
-
DreamZero: "World Action Models are Zero-shot Policies", arXiv 2026.
-
GigaWorld-Policy: "GigaWorld-Policy: An Efficient Action-Centered World-Action Model", arXiv 2026.
-
X-WAM: "Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising", arXiv 2026.
-
[NEW] HiMem-WAM: "HiMem-WAM: Hierarchical Memory-Gated World Action Models for Robotic Manipulation", arXiv 2026.
-
FLARE: "FLARE: Robot Learning with Implicit World Modeling", CoRL 2025.
-
FRAPPE: "FRAPPE: Infusing World Modeling into Generalist Policies via Multiple Future Representation Alignment", arXiv 2026.
-
DUST: "Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model", arXiv 2025.
-
UD-VLA: "Unified Diffusion VLA: Vision-Language-Action Model via Joint Discrete Denoising Diffusion Process", arXiv 2025.
-
Motus: "Motus: A Unified Latent Action World Model", arXiv 2025.
-
CoVAR: "CoVAR: Co-generation of Video and Action for Robotic Manipulation via Multi-Modal Diffusion", arXiv 2025.
-
LingBot-VA: "Causal World Modeling for Robot Control", arXiv 2026.
-
[NEW] Flash-WAM: "Flash-WAM: Modality-Aware Distillation for World Action Models", arXiv 2026.
-
LDA-1B: "LDA-1B: Scaling Latent Dynamics Action Model via Universal Embodied Data Ingestion", arXiv 2026.
-
AdaWorldPolicy: "AdaWorldPolicy: World-Model-Driven Diffusion Policy with Online Adaptive Learning for Robotic Manipulation", arXiv 2026.
-
AIM: "AIM: Intent-Aware Unified world action Modeling with Spatial Value Maps", arXiv 2026.
-
DexWorldModel: "DexWorldModel: Causal Latent World Modeling towards Automated Learning of Embodied Tasks", arXiv 2026.
-
MotuBrain: "MotuBrain: An Advanced World Action Model for Robot Control", arXiv 2026.
-
Act2Goal: "Act2Goal: From World Model To General Goal-conditioned Policy", arXiv 2025.
-
DiT4DiT: "DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control", arXiv 2026.
-
Fast-WAM: "Fast-WAM: Do World Action Models Need Test-time Future Imagination?", arXiv 2026.
-
WAV: "World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems", arXiv 2026.
-
UVA: "Unified Video Action Model", RSS 2025.
-
PhysGen: "Learning Physics from Pretrained Video Models: A Multimodal Continuous and Sequential World Interaction Models for Robotic Manipulation", arXiv 2026.
-
MotionWAM: "MotionWAM: Towards Foundation World Action Models for Real-Time Humanoid Loco-Manipulation", arXiv 2026.
-
AHA-WAM: "AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routing", arXiv 2026.
-
Dream-Tac: "Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulation", arXiv 2026.
-
Efficient-WAM: "Efficient-WAM: A 1B-Parameter World-Action Model with Low-Cost Future Imagination", arXiv 2026.
-
Making Foresight Actionable: "Making Foresight Actionable: Repurposing Representation Alignment in World Action Models", arXiv 2026.
-
MV-WAM: "MV-WAM: Manifold-Aware World Action Model with Value Augmentation", arXiv 2026.
-
MemoryWAM: "MemoryWAM: Efficient World Action Modeling with Persistent Memory", arXiv 2026.
World Model for VLA
World Model for Imitation Learning
-
DREMA: "Dream to Manipulate: Compositional World Models Empowering Robot Imitation Learning with Imagination", ICLR 2025. [📄 Paper] [🌍 Webpage] [💻 Code]
-
RoboScape: "RoboScape: Physics-informed Embodied World Model", arXiv 2025. [📄 Paper]
-
Ctrl-World: "Ctrl-World: A Controllable Generative World Model for Robot", ICLR 2026. [📄 Paper] [🌍 Webpage] [💻 Code]
World Model for Reinforcement Learning
-
PlaNet: "Learning Latent Dynamics for Planning from Pixels", ICML 2019. [📄 Paper] [🌍 Webpage] [💻 Code]
-
Dream to Control: "Dream to Control: Learning Behaviors by Latent Imagination", ICLR 2020. [📄 Paper] [🌍 Webpage] [💻 Code]
-
Dreamer V2: "Mastering Atari with Discrete World Models", ICLR 2021. [📄 Paper] [🌍 Webpage] [💻 Code]
-
DayDreamer: "DayDreamer: World Models for Physical Robot Learning", CoRL 2022. [📄 Paper] [🌍 Webpage] [💻 Code]
-
Dreamer V3: "Mastering Diverse Domains through World Models", Nature 2025. [📄 Paper] [🌍 Webpage] [💻 Code]
-
MoDem-V2: "MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation", ICRA 2024. [📄 Paper] [🌍 Webpage] [💻 Code]
-
VIPER: "Video Prediction Models as Rewards for Reinforcement Learning", NeurIPS 2023. [📄 Paper] [🌍 Webpage] [💻 Code]
-
Diffusion Reward: "Diffusion Reward: Learning Rewards via Conditional Video Diffusion", ECCV 2024. [📄 Paper] [🌍 Webpage] [💻 Code] [🤗 Model]
-
RWM‑U: "Uncertainty-Aware Robotic World Model Makes Offline Model-Based Reinforcement Learning Work on Real Robots", arXiv 2025. [📄 Paper] [🌍 Webpage] [💻 Code]
-
DiWA: "DiWA: Diffusion Policy Adaptation with World Models", arXiv 2025. [📄 Paper] [🌍 Webpage] [💻 Code]
-
World4RL: "World4RL: Diffusion World Models for Policy Refinement with Reinforcement Learning for Robotic Manipulation", arXiv 2025. [📄 Paper] [🌍 Webpage] [💻 Code]
-
Dreamer 4: "Training Agents Inside of Scalable World Models", arXiv 2025. [📄 Paper] [🌍 Webpage]
-
World-Env: "World-Env: Leveraging World Model as a Virtual Environment for VLA Post-Training", arXiv 2025. [📄 Paper] [💻 Code]
-
VLA-RFT: "VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulators", arXiv 2025. [📄 Paper] [🌍 Webpage] [💻 Code] [🤗 Model]
-
PhysWorl: "Robot Learning from a Physical World Model", arXiv 2025. [📄 Paper] [🌍 Webpage] [💻 Code]
-
WMPO: "WMPO: World Model-based Policy Optimization for Vision-Language-Action Models", arXiv 2025. [📄 Paper] [🌍 Webpage] [💻 Code]
-
SRPO: "SRPO: Self-Referential Policy Optimization for Vision-Language-Action Models", arXiv 2025. [📄 Paper] [💻 Code] [🤗 Model]
-
GenReward: "Goal-Driven Reward by Video Diffusion Models for Reinforcement Learning", arXiv 2025. [📄 Paper] [🌍 Webpage]
-
RoboScape-R: "RoboScape-R: Unified Reward-Observation World Models for Generalizable Robotics Training via RL", arXiv 2025. [📄 Paper]
-
World-Gymnast: "World-Gymnast: Training Robots with Reinforcement Learning in a World Model", arXiv 2026. [📄 Paper] [🌍 Webpage] [💻 Code]
-
RWML: "Reinforcement World Model Learning for LLM-based Agents", arXiv 2026. [📄 Paper]
-
RISE: "RISE: Self-Improving Robot Policy with Compositional World Model", arXiv 2026. [📄 Paper] [💻 Code]
-
WoVR: "WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RL", arXiv 2026. [📄 Paper] [💻 Code] [🤗 Model]
World Model for Evaluation
-
WorldEval: "WorldEval: World Model as Real-World Robot Policies Evaluator", arXiv 2025. [📄 Paper] [🌍 Webpage] [💻 Code]
-
WorldGym: "WorldGym: World Model as An Environment for Policy Evaluation", arXiv 2025. [📄 Paper] [🌍 Webpage] [💻 Code]
-
Veo: "Evaluating Gemini Robotics Policies in a Veo World Simulator", arXiv 2025. [📄 Paper] [🌍 Webpage]
-
Interactive World Simulator: "Interactive World Simulator for Robot Policy Training and Evaluation", arXiv 2026. [📄 Paper] [🌍 Webpage] [💻 Code] [🤗 Model]
-
dWorldEval: "dWorldEval: Scalable Robotic Policy Evaluation via Discrete Diffusion World Model", arXiv 2026. [📄 Paper] [🌍 Webpage]
Training Data
🤖 Robot-Centric
| Paper | Released | Links |
|---|---|---|
| QT-Opt - QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation | 2018-06 | 📄 Paper · 🌍 Web · 📦 Dataset |
| MIME - Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitation | 2018-10 | 📄 Paper · 🌍 Web · 📦 Dataset |
| RoboNet - RoboNet: Large-Scale Multi-Robot Learning | 2019-10 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| RoboTurk - Scaling Robot Supervision to Hundreds of Hours with RoboTurk: Robotic Manipulation Dataset through Human Reasoning and Dexterity | 2019-11 | 📄 Paper · 🌍 Web · 📦 Dataset |
| Bridge - Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets | 2021-09 | 📄 Paper · 🌍 Web · 📦 Dataset |
| MT-Opt - MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale | 2021-04 | 📄 Paper · 🌍 Web · 📦 Dataset |
| BC-Z - BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning | 2022-02 | 📄 Paper · 🌍 Web · 📦 Dataset |
| Language-Table - Interactive Language: Talking to Robots in Real Time | 2022-10 | 📄 Paper · 🌍 Web · 📦 Dataset |
| RT-1 - RT-1: Robotics Transformer for Real-World Control at Scale | 2022-12 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| BridgeData V2 - BridgeData V2: A Dataset for Robot Learning at Scale | 2023-08 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| Jaco-Play - CLVR Jaco Play Dataset | 2023-04 | 💻 Code· 📦 Dataset |
| Cable-Routing-Dataset - Multi-Stage Cable Routing through Hierarchical Imitation Learning | 2023-07 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| RH20T - RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot | 2023-07 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| OXE - Open X-Embodiment: Robotic Learning Datasets and RT-X Models | 2023-10 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| DROID - DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset | 2024-03 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| RH20T-P - RH20T-P: A Primitive-Level Robotic Dataset Towards Composable Generalization Agents | 2024-03 | 📄 Paper · 🌍 Web · 📦 Dataset |
| RoboMIND - RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation | 2024-12 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| ARIO - All Robots in One: A New Standard and Unified Dataset for Versatile, General-Purpose Embodied Agents | 2024-08 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| RoboData - RoboTron-Mani: All-in-One Multimodal Large Model for Robotic Manipulation | 2024-12 | 📄 Paper · 📦 Dataset · 💻 Code |
| DexCap - DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation | 2024-03 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| FuSe - Beyond Sight: Finetuning Generalist Robot Policies with Heterogeneous Sensors via Language Grounding | 2025-01 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| AgiBot World Colosseo - AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems | 2025-03 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| REASSEMBLE - REASSEMBLE: A Multimodal Dataset for Contact-rich Robotic Assembly and Disassembly | 2025-02 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| OmniAction - RoboOmni: Proactive Robot Manipulation in Omni-modal Context | 2025-10 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code · |
| UnifoLM-WBT - UnifoLM-WBT-Dataset | 2026-03 | 📦 Dataset |
🖐️ UMI
| Paper | Released | Links |
|---|---|---|
| UMI - Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots | 2024-02 | 📄 Paper · 🌍 Web · 💻 Code· 📦 Dataset |
| FastUMI - FastUMI: A Scalable and Hardware-Independent Universal Manipulation Interface with Dataset | 2024-09 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| FastUMI-100K - FastUMI-100K: Advancing Data-driven Robotic Manipulation with a Large-scale UMI-style Dataset | 2025-10 | 📄 Paper · 💻 Code |
| RealOmin - 10Kh-RealOmin-OpenData: A Large-Scale Real-World Manipulation Dataset | 2026-01 | 🌍 Web · 📦 Dataset · 💻 Code |
| Hoi! - Hoi! -- A Multimodal Dataset for Force-Grounded, Cross-View Articulated Manipulation | 2025-12 | 📄 Paper · 🌍 Web · 📦 Dataset |
| DexUMI - DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation | 2025-05 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
🖥️ Simulation
| Paper | Released | Links |
|---|---|---|
| MimicGen - MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations | 2023-10 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| ManiSkill2 - ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills | 2023-02 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| RoboCasa - RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots | 2024-06 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| RoboTwin - RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins | 2025-04 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| DexMimicGen - DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learning | 2024-10 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| QUARD-Auto - GeRM: A Generalist Robotic Model with Mixture-of-experts for Quadruped Robot | 2024-03 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| TesserAct - TesserAct: Learning 4D Embodied World Models | 2025-04 | 📄 Paper · 🌍 Web · 💻 Code |
| RoboCerebra - RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation | 2025-06 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| SynGrasp-1B - GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data | 2025-05 | 📄 Paper · 🌍 Web · 💻 Code |
| RoboTwin 2.0 - RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation | 2025-06 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| TLA - TLA: Tactile-Language-Action Model for Contact-Rich Manipulation | 2025-03 | 📄 Paper · 🌍 Web |
| InternVLA-M1 - InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policy | 2025-10 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| InternData-A1 - InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulation | 2026-01 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
👁️ Human / Egocentric
| Paper | Released | Links |
|---|---|---|
| SSv2 - The "something something" video database for learning and evaluating visual common sense | 2017-06 | 📄 Paper · 📦 Dataset |
| EPIC-KITCHENS - Scaling Egocentric Vision: The EPIC-KITCHENS Dataset | 2018-04 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| HowTo100M - HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips | 2019-06 | 📄 Paper · 🌍 Web · 📦 Dataset |
| Kinetics-700 - A Short Note on the Kinetics-700 Human Action Dataset | 2019-07 | 📄 Paper · 📦 Dataset · 💻 Code |
| EGTEA Gaze+ - In the Eye of the Beholder: Gaze and Actions in First Person Video | 2020-06 | 📄 Paper · 🌍 Web · 📦 Dataset |
| Ego4D - Ego4D: Around the World in 3,000 Hours of Egocentric Video | 2021-10 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| H2O - H2O: Two Hands Manipulating Objects for First Person Interaction Recognition | 2021-04 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| HOI4D - HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction | 2022-03 | 📄 Paper · 🌍 Web · 📦 Dataset |
| Assembly101 - Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities | 2022-03 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| EgoPAT3D - Egocentric Prediction of Action Target in 3D | 2022-03 | 📄 Paper · 📦 Dataset · 💻 Code |
| ARCTIC - ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation | 2022-04 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| HoloAssist - HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World | 2023-09 | 📄 Paper · 🌍 Web · 📦 Dataset |
| Ego-Exo4D - Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives | 2023-11 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| TACO - TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understanding | 2024-01 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| Aria Everyday Activities - Aria Everyday Activities Dataset | 2024-02 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| OAKINK2 - OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completion | 2024-03 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| Nymeria - Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wild | 2024-06 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| COM Kitchens - COM Kitchens: An Unedited Overhead-view Video Dataset as a Vision-Language Benchmark | 2024-08 | 📄 Paper · 📦 Dataset · 💻 Code |
| EgoVid-5M - EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Video Generation | 2024-11 | 📄 Paper · 🌍 Web · 📦 Dataset |
| EgoMimic - EgoMimic: Scaling Imitation Learning via Egocentric Video | 2024-10 | 📄 Paper · 🌍 Web · 📦 Dataset |
| HOT3D - HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos | 2024-11 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| Egocentric-10K - Egocentric-10K | 2025-11 | 🌍 Web · 📦 Dataset |
| DreamDojo - DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos | 2026-02 | 📄 Paper · 🌍 Web · 💻 Code |
| PH²D - Humanoid Policy ~ Human Policy | 2025-03 | 📄 Paper · 🌍 Web · 📦 Dataset |
| Humanoid Everyday - Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulation | 2025-10 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| IndEgo - IndEgo: A Dataset of Industrial Scenarios and Collaborative Work for Egocentric Assistants | 2025-11 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| PLAICraft - PLAICraft: Large-Scale Time-Aligned Vision-Speech-Action Dataset for Embodied AI | 2025-05 | 📄 Paper · 🌍 Web · 📦 Dataset |
| HD-EPIC - HD-EPIC: A Highly-Detailed Egocentric Video Dataset | 2025-02 | 📄 Paper · 🌍 Web · 📦 Dataset |
| UniHand - Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videos | 2025-07 | 📄 Paper · 🌍 Web · 📦 Dataset · 💻 Code |
| Ego-Centric Human Manipulation Dataset - EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos | 2025-07 | 📄 Paper · 🌍 Web · 📦 Dataset |
| Kaiwu - Kaiwu: A Multimodal Manipulation Dataset and Framework for Robot Learning and Human-Robot Interaction | 2025-03 | 📄 Paper · 📦 Dataset |
| EgoDex - EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video | 2025-05 | 📄 Paper · 📦 Dataset · 💻 Code |
Evaluation
🌐 World Model — Visual Fidelity
| Paper | Released | Links |
|---|---|---|
| PSNR / SSIM - Image Quality Assessment: From Error Visibility to Structural Similarity | 2004 | 📄 Paper · 🌍 Web · 💻 Code |
| LPIPS - The Unreasonable Effectiveness of Deep Features as a Perceptual Metric | 2018-01 | 📄 Paper · 🌍 Web · 💻 Code |
| DreamSim - DreamSim: Learning New Dimensions of Human Visual Similarity Using Synthetic Data | 2023-06 | 📄 Paper · 🌍 Web · 💻 Code |
| DINOv2 - DINOv2: Learning Robust Visual Features without Supervision | 2023-04 | 📄 Paper · 🌍 Web · 💻 Code |
| FVD - Towards Accurate Generative Models of Video: A New Metric & Challenges | 2018-12 | 📄 Paper · 💻 Code |
🌐 World Model — Physical Commonsense
| Paper | Released | Links |
|---|---|---|
| VideoPhy - VideoPhy: Evaluating Physical Commonsense for Video Generation | 2024-06 | 📄 Paper · 🌍 Web · 💻 Code · 📦 Dataset |
| PhyGenBench - Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation | 2024-10 | 📄 Paper · 🌍 Web · 💻 Code |
| VBench-2.0 - VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness | 2025-03 | 📄 Paper · 🌍 Web · 💻 Code · 📦 Dataset |
| WorldModelBench - WorldModelBench: Judging Video Generation Models as World Models | 2025-02 | 📄 Paper · 🌍 Web · 💻 Code |
| Physics-IQ - Do Generative Video Models Understand Physical Principles? | 2025-01 | 📄 Paper · 🌍 Web · 💻 Code |
| WorldScore - WorldScore: A Unified Evaluation Benchmark for World Generation | 2025-04 | 📄 Paper · 🌍 Web · 💻 Code |
| EWMBench - EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models | 2025-05 | 📄 Paper · 💻 Code |
🌐 World Model — Action Plausibility
| Paper | Released | Links |
|---|---|---|
| WorldSimBench - WorldSimBench: Towards Video Generation Models as World Simulators | 2024-10 | 📄 Paper · 🌍 Web · 💻 Code |
| Wow, wo, val! - A Comprehensive Embodied World Model Evaluation Turing Test | 2026-01 | 📄 Paper |
🤖 Action Policy — General
| Paper | Released | Links |
|---|---|---|
| MetaWorld - MetaWorld: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning | 2019-10 | 📄 Paper · 🌍 Web · 💻 Code |
| RLBench - RLBench: The Robot Learning Benchmark & Learning Environment | 2019-09 | 📄 Paper · 🌍 Web · 💻 Code |
| Robomimic - What Matters in Learning from Offline Human Demonstrations for Robot Manipulation | 2021-08 | 📄 Paper · 🌍 Web · 💻 Code · 📦 Dataset |
| Franka Kitchen - Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning | 2019-10 | 📄 Paper · 🌍 Web · 💻 Code |
| ManiSkill - ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations | 2021-07 | 📄 Paper · 🌍 Web · 💻 Code |
| ManiSkill2 - ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills | 2023-02 | 📄 Paper · 🌍 Web · 💻 Code · 📦 Dataset |
| ManiSkill3 - ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI | 2024-10 | 📄 Paper · 🌍 Web · 💻 Code · 📦 Dataset |
| RoboCasa - RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots | 2024-06 | 📄 Paper · 🌍 Web · 💻 Code |
| CALVIN - CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks | 2021-12 | 📄 Paper · 🌍 Web · 💻 Code |
| VIMAbench - VIMA: General Robot Manipulation with Multimodal Prompts | 2022-10 | 📄 Paper · 🌍 Web · 💻 Code |
| VLMbench - VLMbench: A Compositional Benchmark for Vision-and-Language Manipulation | 2022-06 | 📄 Paper · 💻 Code |
| LIBERO - LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning | 2023-06 | 📄 Paper · 🌍 Web · 💻 Code |
| LIBERO-Plus - LIBERO-Plus: In-Depth Robustness Analysis of Vision-Language-Action Models | 2025-10 | 📄 Paper · 🌍 Web · 💻 Code · 📦 Dataset |
| LIBERO-PRO - LIBERO-PRO: Towards Robust and Fair Evaluation of VLA Models beyond Memorization | 2025-10 | 📄 Paper · 🌍 Web · 💻 Code · 📦 Dataset |
| LIBERO-X - LIBERO-X: Robustness Litmus for Vision-Language-Action Models | 2026-02 | 📄 Paper· 🌍 Web · 💻 Code |
| COLOSSEUM - THE COLOSSEUM: A Benchmark for Evaluating Generalization for Robotic Manipulation | 2024-02 | 📄 Paper · 🌍 Web · 💻 Code |
| AGNOSTOS - Exploring the Limits of Vision-Language-Action Manipulations in Cross-Task Generalization | 2025-05 | 📄 Paper · 🌍 Web · 💻 Code |
| RoboEval - RoboEval: Where Robotic Manipulation Meets Structured and Scalable Evaluation | 2025-07 | 📄 Paper · 🌍 Web· 💻 Code |
| RoboVerse - RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning | 2025-04 | 📄 Paper · 🌍 Web · 💻 Code |
| PolaRiS - PolaRiS: Scalable Real-to-Sim Evaluations for Generalist Robot Policies | 2025-12 | 📄 Paper · 🌍 Web · 💻 Code · 📦 Dataset |
| RoboMME - RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies | 2026-03 | 📄 Paper · 🌍 Web · 💻 Code |
| GenManip - GenManip: LLM-Driven Simulation for Generalizable Instruction-Following Manipulation | 2025-06 | 📄 Paper · 🌍 Web |
| VLABench - VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks | 2024-12 | 📄 Paper · 💻 Code |
| RoboSuite - Robosuite: A Modular Simulation Framework and Benchmark for Robot Learning | 2020-09 | 📄 Paper · 🌍 Web · 💻 Code |
| RoboLab - RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies | 2026-04 | 📄 Paper |
| SimplerEnv - Evaluating Real-World Robot Manipulation Policies in Simulation | 2024-05 | 📄 Paper · 🌍 Web · 💻 Code |
| ARNOLD - ARNOLD: A Benchmark for Language-Grounded Task Learning with Continuous States in Realistic 3D Scenes | 2023-04 | 📄 Paper · 🌍 Web · 💻 Code |
| GemBench - Towards Generalizable Vision-Language Robotic Manipulation: A Benchmark and LLM-Guided 3D Policy | 2024-10 | 📄 Paper · 🌍 Web · 💻 Code |
🤖 Action Policy — Bimanual and Humanoid Form
| Paper | Released | Links |
|---|---|---|
| RoboTwin - RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins | 2025-04 | 📄 Paper · 🌍 Web · 💻 Code |
| BiGym - BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmark | 2024-07 | 📄 Paper · 🌍 Web · 💻 Code |
| HumanoidBench - HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation | 2024-03 | 📄 Paper · 🌍 Web · 💻 Code |
| HumanoidGen - HumanoidGen: Data Generation for Bimanual Dexterous Manipulation via LLM Reasoning | 2025-07 | 📄 Paper · 🌍 Web · 💻 Code |
🤖 Action Policy — Mobile Manipulation
| Paper | Released | Links |
|---|---|---|
| ManipulaTHOR - ManipulaTHOR: A Framework for Visual Object Manipulation | 2021-04 | 📄 Paper · 🌍 Web · 💻 Code |
| HomeRobot - HomeRobot: Open-Vocabulary Mobile Manipulation | 2023-06 | 📄 Paper · 🌍 Web · 💻 Code |
| BEHAVIOR-1K - BEHAVIOR-1K: A Benchmark for Embodied AI with 1,000 Everyday Activities and Realistic Simulation | 2024-03 | 📄 Paper · 🌍 Web · 💻 Code |
🤖 Action Policy — Contact and Deformation Manipulation
| Paper | Released | Links |
|---|---|---|
| SoftGym - SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation | 2020-11 | 📄 Paper· 🌍 Web · 💻 Code |
| PlasticineLab - PlasticineLab: A Soft-Body Manipulation Benchmark with Differentiable Physics | 2021-04 | 📄 Paper · 🌍 Web · 💻 Code |
| DaXBench - DaXBench: Benchmarking Deformable Object Manipulation with Differentiable Physics | 2022-10 | 📄 Paper · 🌍 Web · 💻 Code |
| TacSL - TacSL: A Library for Visuotactile Sensor Simulation and Learning | 2024-08 | 📄 Paper · 🌍 Web · 💻 Code |
| ManiFeel - ManiFeel: Benchmarking and Understanding Visuotactile Manipulation Policy Learning | 2025-05 | 📄 Paper · 🌍 Web· 💻 Code |
🤖 Action Policy — Real-Device
| Paper | Released | Links |
|---|---|---|
| RoboArena - RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies | 2025-06 | 📄 Paper · 🌍 Web · 💻 Code |
| RoboChallenge - RoboChallenge: Large-Scale Real-Robot Evaluation of Embodied Policies | 2025-10 | 📄 Paper · 🌍 Web· 💻 Code |
| ManipArena - ManipArena: Comprehensive Real-World Evaluation of Reasoning-Oriented Generalist Robot Manipulation | 2026-03 | 📄 Paper · 🌍 Web · 💻 Code |
🌟 Star History
🙏 Acknowledgements
We thank the AllenAI VLA Evaluation Harness team for curating and maintaining the VLA leaderboard data that supports our leaderboard.
👋 Citation
If you find this survey or repository helpful for your research, please consider citing our paper:
@article{wang2026world,
title={World Action Models for Generalist Robotics: From Next Token Prediction to Next State Synthesis},
author={Wang, Siyin and Shi, Junhao and Fu, Zhaoyang and He, Xinzhe and Liu, Feihong and Yang, Chenchen and Zhou, Yikang and Fei, Zhaoye and Gong, Jingjing and Fu, Jinlan and Shou, Mike Zheng and Huang, Xuanjing and Qiu, Xipeng and Jiang, Yu-Gang},
year={2026}
}