Awesome VLM-based VLA for Robotic Manipulation

April 3, 2026 ยท View on GitHub

Awesome VLM-based VLA for Robotic Manipulation

arXiv Lab Contribution Welcome GitHub star chart License

๐Ÿ› ๏ธ We're still cooking โ€” Stay tuned!๐Ÿ› ๏ธ
โญ Give us a star if you like it! โญ
โœจIf you find this work useful for your research, please kindly cite our paper.โœจ
๐Ÿ“ข Update: Under Major Revision at IEEE TPAMI.

image info

๐Ÿ”ฅ Large VLM-based Vision-Language-Action (VLA) models have recently emerged as a transformative paradigm for robotic manipulation by tightly coupling perception, language understanding, and action generation. Built upon large Vision-Language Models (VLMs), they enable robots to interpret natural language instructions, perceive complex environments, and perform diverse manipulation tasks with strong generalization.

๐Ÿ“ We present the first systematic survey on large VLM-based VLA models for robotic manipulation. This repository serves as the companion resource to our survey: "Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey", and includes all the research papers, benchmarks, and resources reviewed in the paper, organized for easy access and reference.

๐Ÿ“Œ We will keep updating this repository with newly published works to reflect the latest progress in the field.

Table of Contents

Monolithic Models

Single-System

YearVenuePaperWebsiteCode
2023CoRLRT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control๐ŸŒ-
2023ICRART-2-X: Open X-Embodiment: Robotic Learning Datasets and RT-X Models๐ŸŒ-
2023NeurIPSRoboFlamingo: Vision-Language Foundation Models as Effective Robot Imitators๐ŸŒ๐Ÿ’ป
2023ICMLLEO Agent: An Embodied Generalist Agent in 3D World๐ŸŒ๐Ÿ’ป
2024NeurIPSRoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulation๐ŸŒ๐Ÿ’ป
2024CoRLOpenVLA: An Open-Source Vision-Language-Action Model๐ŸŒ๐Ÿ’ป
2024CoRLECOT-Lite: Robotic Control via Embodied Chain-of-Thought Reasoning๐ŸŒ๐Ÿ’ป
2024ICRAReVLA: Reverting Visual Domain Limitation of Robotic Foundation Models๐ŸŒ-
2024NeurIPSDeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution-๐Ÿ’ป
2024ICLRTraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policies๐ŸŒ๐Ÿ’ป
2025ICRAFuSe: Beyond Sight: Finetuning Generalist Robot Policies with Heterogeneous Sensors via Language Grounding๐ŸŒ๐Ÿ’ป
2025CVPRUniAct: Universal Actions for Enhanced Embodied Foundation Models๐ŸŒ๐Ÿ’ป
2025arXivSpatialVLA: Exploring Spatial Representations for Visual-Language-Action Model๐ŸŒ๐Ÿ’ป
2025ICMLUP-VLA: A Unified Understanding and Prediction Model for Embodied Agent-๐Ÿ’ป
2025ICLRVLAS: Vision-Language-Action Model with Speech Instructions for Customized Robot Manipulation--
2025arXivOpenVLA-OFT: Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success๐ŸŒ๐Ÿ’ป
2025arXivPD-VLA: Accelerating Vision-Language-Action Model Integrated with Action Chunking via Parallel Decoding--
2025arXivHybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model๐ŸŒ๐Ÿ’ป
2025arXivMoLe-VLA: Dynamic Layer-Skipping Vision-Language-Action Model via Mixture-of-Layers for Efficient Robot Manipulation๐ŸŒ๐Ÿ’ป
2025CVPRCoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models๐ŸŒ-
2025arXivNORA: A Small Open-Sourced Generalist Vision-Language-Action Model for Embodied Tasks๐ŸŒ๐Ÿ’ป
2025arXivVTLA: Vision-Tactile-Language-Action Model with Preference Learning for Insertion Manipulation๐ŸŒ-
2025arXivOE-VLA: Unveiling the Potential of Vision-Language-Action Models with Open-Ended Multimodal Instructions--
2025arXivReFineVLA: Reasoning-Aware Teacher-Guided Transfer Fine-Tuning--
2025arXivFLashVLA: Think Twice, Act Once: Token-Aware Compression and Action Reuse for Efficient Inference in Vision-Language-Action Models--
2025arXivLoHoVLA: A Unified Vision-Language-Action Model for Long-Horizon Embodied Tasks--
2025arXivBitVLA: 1-Bit Vision-Language-Action Models for Robotics Manipulation-๐Ÿ’ป
2025arXivBridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Models๐ŸŒ๐Ÿ’ป
2025arXivUniVLA: Unified Vision-Language-Action Model๐ŸŒ๐Ÿ’ป
2025arXivWorldVLA: Towards Autoregressive Action World Model-๐Ÿ’ป
2025arXiv4D-VLA: Spatiotemporal Vision-Language-Action Pretraining with Cross-Scene Calibration-๐Ÿ’ป
2025ICCVVQ-VLA: Improving Vision-Language-Action Models via Scaling Vector-Quantized Action Tokenizers๐ŸŒ๐Ÿ’ป
2025arXivVOTE: Vision-Language-Action Optimization with Trajectory Ensemble Voting--
2025arXivSpecVLA: Speculative Decoding for Vision-Language-Action Models with Relaxed Acceptance--
2025arXivST-VLA: Spatial Traces: Enhancing VLA Models with Spatial-Temporal Understanding๐ŸŒ-
2025arXivDiscrete Diffusion VLA: Bringing Discrete Diffusion to Action Decoding in Vision-Language-Action Policies--
2025arXivLLaDA-VLA: Vision Language Diffusion Action Models๐ŸŒ-
2025arXivOccVLA: Vision-Language-Action Model with Implicit 3D Occupancy Supervision--
2025arXivTwinVLA: Data-Efficient Bimanual Manipulation with Twin Single-Arm Vision-Language-Action Models๐ŸŒ-
2025arXivSTARE-VLA: Progressive Stage-Aware Reinforcement for Fine-Tuning Vision-Language-Action Models๐ŸŒ-

Dual-System

YearVenuePaperWebsiteCode
2024arXivA dual process vla: efficient robotic manipulation leveraging vlm--
2024arXivTowards synergistic, generalized, and efficient dual-system for robotic manipulation๐ŸŒ-
2024IROSFrom llms to actions: latent codes as bridges in hierarchical robot control--
2025arXivGr00t n1: an open foundation model for generalist humanoid robots--
2024arXivCogact: a foundational vision-language-action model for synergizing cognition and action in robotic manipulation๐ŸŒ๐Ÿ’ป
2024CoRLHirt: enhancing robotic control with hierarchical robot transformers--
2025arXivGraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data๐ŸŒ๐Ÿ’ป
2025arXivFast-in-slow: a dual-system foundation model unifying fast manipulation within slow reasoning๐ŸŒ๐Ÿ’ป
2025arXivOpenhelix: A short survey, empirical analysis, and open-source dual-system vla model for robotic manipulation๐ŸŒ๐Ÿ’ป
2025arXivChatvla: unified multimodal understanding and robot control with vision-language-action model๐ŸŒ๐Ÿ’ป
2025arXivChatvla-2: vision-language-action model with open-world embodied reasoning from pretrained knowledge๐ŸŒ-
2025ICMLDiffusionvla: scaling robot foundation models via unified diffusion and autoregression๐ŸŒ๐Ÿ’ป
2025arXivTrivla: a unified triple-system-based unified vision-language-action model for general robot control๐ŸŒ-
2025arXivInformation-theoretic graph fusion with vision-language-action model for policy reasoning and dual robotic control--
2025arXivRationalvla: a rational vision-language-action model with dual system๐ŸŒ-
2025ICCVVq-vla: improving vision-language-action models via scaling vector-quantized action tokenizers๐ŸŒ๐Ÿ’ป
2025RA-LTinyvla: towards fast, data-efficient vision-language-action models for robotic manipulation๐ŸŒ๐Ÿ’ป
2025RSSฯ€0: A vision-language-action flow model for general robot control๐ŸŒ-
2025RSSFast: efficient action tokenization for vision-language-action models๐ŸŒ-
2025arXivฯ€0.5: a vision language-action model with open-world generalization๐ŸŒ-
2025arXivKnowledge insulating vision-language-action models: train fast, run fast, generalize better๐ŸŒ-
2025arXivForcevla: enhancing vla models with a force-aware moe for contact-rich manipulation๐ŸŒ-
2025arXivSmolvla: a vision-language-action model for affordable and efficient robotics-๐Ÿ’ป
2025arXivOnetwovla: a unified vision-language-action model with adaptive reasoning๐ŸŒ๐Ÿ’ป
2025arXivTactile-vla: unlocking vision-language-action modelโ€™s physical knowledge for tactile generalization--
2025arXivGr-3 technical report๐ŸŒ-
2025arXivVilla-x: enhancing latent action modeling in vision-language-action models๐ŸŒ๐Ÿ’ป
2025arXivThinkAct: Vision-Language-Action Reasoning via Reinforced Visual Latent Planning๐ŸŒ-
2025arXivF1: A Vision-Language-Action Model Bridging Understanding and Generation to Actions๐ŸŒ๐Ÿ’ป
2025arXiviFlyBot-VLA๐ŸŒ-
2025arXiviFlyBot-VLA Technical Report๐ŸŒ-
2025arXivEvo-1: Lightweight Vision-Language-Action Model with Preserved Semantic Alignment๐ŸŒ๐Ÿ’ป
2025arXivNORA-1.5: A Vision-Language-Action Model Trained using World Model- and Action-based Preference Rewards๐ŸŒ๐Ÿ’ป
2025arXivฯ€*0.6: a VLA that Learns from Experience๐ŸŒ-
2025arXivManualVLA: A Unified VLA Model for Chain-of-Thought Manual Generation and Robotic Manipulation๐ŸŒ-
2025arXivMETIS: Multi-Source Egocentric Training for Integrated Dexterous Vision-Language-Action Model๐ŸŒ-

Hierarchical Models

Planner Only

YearVenuePaperWebsiteCode
2023ICMLPaLM-E: An embodied multimodal language model๐ŸŒ-
2023arXivViLa: Look before you leap: Unveiling the power of GPT-4V in robotic vision-language planning๐ŸŒ-
2024CVPRMoManipVLA: Transferring vision-language-action models for general mobile manipulation๐ŸŒ-
2024CoRLRoboPoint: A vision-language model for spatial affordance prediction in robotics๐ŸŒ๐Ÿ’ป
2025arXivManipLVM-R1: Reinforcement learning for reasoning in embodied manipulation with large vision-language models--
2025arXivEmbodied-Reasoner: Synergizing visual search, reasoning, and action for embodied interactive tasks๐ŸŒ๐Ÿ’ป
2025arXivReinforced Planning: Solving long-horizon tasks via imitation and reinforcement learning-๐Ÿ’ป
2025ICRAChain-of-Modality: Learning manipulation programs from multimodal human videos with vision-language-models๐ŸŒ-
2025arXivEmbodied-R: Collaborative framework for activating embodied spatial reasoning in foundation models via reinforcement learning๐ŸŒ๐Ÿ’ป
2025CVPRRoVI: Robotic Visual Instruction๐ŸŒ-
2025arXivReLEP: Long-horizon embodied planning with implicit logical inference and hallucination mitigation--
2025CVPRRoboBrain: A unified brain model for robotic manipulation from abstract to concrete๐ŸŒ๐Ÿ’ป

Planner + Policy

YearVenuePaperWebsiteCode
2023arXivInstruct2Act: Mapping multi-modality instructions to robotic actions with large language model-๐Ÿ’ป
2023CoRLVoxPoser: Composable 3D value maps for robotic manipulation with language models๐ŸŒ๐Ÿ’ป
2024CVPRSkillDiffuser: Interpretable hierarchical planning via skill abstractions in diffusion-based task execution๐ŸŒ๐Ÿ’ป
2024arXivRoboMatrix: A skill-centric hierarchical framework for scalable robot task planning and execution in open-world-๐Ÿ’ป
2024CoRLRT-Affordance: Reasoning about robotic manipulation with affordances๐ŸŒ-
2024CoRLLLARVA: Vision-action instruction tuning enhances robot learning๐ŸŒ๐Ÿ’ป
2024CVPRMALMM: Multi-agent large language models for zero-shot robotics manipulation๐ŸŒ๐Ÿ’ป
2024arXivRT-H: Action Hierarchies Using Language๐ŸŒ-
2024CoRLReKep: Spatio-temporal reasoning of relational keypoint constraints for robotic manipulation๐ŸŒ๐Ÿ’ป
2025ICLRHAMSTER: Hierarchical action models for open-world robot manipulation๐ŸŒ๐Ÿ’ป
2025ICMLHiRobot: Open-ended instruction following with hierarchical vision-language-action models๐ŸŒ-
2025arXivAgentic Robot: A brain-inspired framework for vision-language-action models in embodied agents๐ŸŒ๐Ÿ’ป
2025arXivDexVLA: Vision-language model with plug-in diffusion expert for general robot control๐ŸŒ๐Ÿ’ป
2025arXivPointVLA: Injecting the 3D world into vision-language-action models๐ŸŒ-
2025arXivA0: An affordance-aware hierarchical model for general robotic manipulation๐ŸŒ๐Ÿ’ป
2025arXivFrom seeing to doing: Bridging reasoning and decision for robotic manipulation๐ŸŒ๐Ÿ’ป
2025ICCVRoBridge: A hierarchical architecture bridging cognition and execution for general robotic manipulation๐ŸŒ๐Ÿ’ป
2025arXivRoboCerebra: A large-scale benchmark for long-horizon robotic manipulation evaluation๐ŸŒ-
2025arXivฯ€0.5: A vision-language-action model with open-world generalization๐ŸŒ-
2025arXivDexGraspVLA: A vision-language-action framework towards general dexterous grasping๐ŸŒ๐Ÿ’ป
2025arXivHiBerNAC: Hierarchical brain-emulated robotic neural agent collective for disentangling complex manipulation--
2025arXivRobix: A Unified Model for Robot Interaction, Reasoning and Planning๐ŸŒ-

Other Advanced Field

Reinforcement Learning-based Methods

YearVenuePaperWebsiteCode
2025ICLR(Workshop)GRAPE: Generalizing Robot Policy via Preference Alignment๐ŸŒ๐Ÿ’ป
2025arXivVla-rl: Towards masterful and general robotic manipulation with scalable reinforcement learning-๐Ÿ’ป
2025RSSReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrations๐ŸŒ-
2025RSSConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy๐ŸŒ๐Ÿ’ป
2025RSSRLDG: Robotic Generalist Policy Distillation via Reinforcement Learning๐ŸŒ-
2025arXivTGRPO: Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimization--
2025ICRAImproving Vision-Language-Action Model with Online Reinforcement Learning--
2025CVPR(Workshop)Interactive Postโ€‘Training for Visionโ€‘Languageโ€‘Action Models๐ŸŒ๐Ÿ’ป
2025arXivEmbodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulation๐ŸŒ๐Ÿ’ป
2025arXivVLAC: A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learning๐ŸŒ๐Ÿ’ป
2025arXivSimpleVLA-RL: Scaling VLA Training via Reinforcement Learning-๐Ÿ’ป

Training-Free Methods

YearVenuePaperWebsiteCode
2025arXivVLAโ€‘Cache: Towards Efficient Visionโ€‘Languageโ€‘Action Model via Adaptive Token Caching in Robotic Manipulation-๐Ÿ’ป
2025arXivAccelerating Vision-Language-Action Model Integrated with Action Chunking via Parallel Decoding--
2025arXivThink Twice, Act Once: Tokenโ€‘Aware Compression and Action Reuse for Efficient Inference in Visionโ€‘Languageโ€‘Action Models--
2025arXivEfficientVLA: Trainingโ€‘Free Acceleration and Compression for Visionโ€‘Languageโ€‘Action Models--
2025arXivSP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration--
2025arXivBlock-wise Adaptive Caching for Accelerating Diffusion Policy--
2025RSSFAST: Efficient Action Tokenization for Vision-Language-Action Models๐ŸŒ-
2025arXivReal-Time Execution of Action Chunking Flow Policies๐ŸŒ-
2025arXivSpecPrune-VLA: Accelerating Vision-Language-Action Models via Action-Aware Self-Speculative Pruning--

Learning from Human Videos

YearVenuePaperWebsiteCode
2024ICML3Dโ€‘VLA: Aย 3D Visionโ€‘Languageโ€‘Action Generative World Model๐ŸŒ๐Ÿ’ป
2024NeurIPSLearning an Actionable Discrete Diffusion Policy via Largeโ€‘Scale Actionless Video Preโ€‘Training๐ŸŒ๐Ÿ’ป
2025CVPRMitigating the Humanโ€‘Robot Domain Discrepancy in Visual Preโ€‘training for Robotic Manipulation๐ŸŒ๐Ÿ’ป
2025RSSUniVLA: Learning to Act Anywhere with Taskโ€‘centric Latent Actions-๐Ÿ’ป
2025ICLRLatent Action Pretraining from Videos๐ŸŒ๐Ÿ’ป
2025arXivHumanoidโ€‘VLA: Towards Universal Humanoid Control with Visual Integration--

World Model-based VLA

YearVenuePaperWebsiteCode
2024CVPRFoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects๐ŸŒ๐Ÿ’ป
2024ICML3Dโ€‘VLA: Aย 3D Visionโ€‘Languageโ€‘Action Generative World Model๐ŸŒ๐Ÿ’ป
2025arXivWorldVLA: Towards Autoregressive Action World Model-๐Ÿ’ป
2025arXivWorld4Omni: A Zeroโ€‘Shot Framework from Image Generation World Model to Robotic Manipulation๐ŸŒ-
2025arXivRobotic Manipulation by Imitating Generated Videos Without Physical Demonstrations๐ŸŒ๐Ÿ’ป
2025arXivVโ€‘JEPAย 2: Selfโ€‘Supervised Video Models Enable Understanding, Prediction and Planning๐ŸŒ๐Ÿ’ป
2025arXivFlowVLA: Thinking in Motion with a Visual Chain of Thought๐ŸŒ-
2025arXivGenie Envisioner: A Unified World Foundation Platform for Robotic Manipulation๐ŸŒ๐Ÿ’ป
2025arXivWoW: Towards a World-omniscient World-model Through Embodied Interaction๐ŸŒ๐Ÿ’ป
2025arXivRoboScape-R: Unified Reward-Observation World Models for Generalizable Robotics Training via RL--

Datasets and Benchmarks

Real-world Robot Datasets

YearVenuePaperWebsiteCodeData
2021CoRLBC-Z: Zero-shot task generalization with robotic imitation learning๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2023RSSRTโ€‘1:โ€ฏRobotics Transformer for Realโ€‘World Control at Scale๐ŸŒ๐Ÿ’ป-
2023CoRLRTโ€‘2: Visionโ€‘Language Foundation Models as Effective Robot Imitators๐ŸŒ๐Ÿ’ป-
2022RSSBridge Data: Boosting Generalization of Robotic Skills with Crossโ€‘Domain Datasets๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2023CoRLBridgeDataโ€ฏV2: A Dataset for Robot Learning at Scale๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2024ICRARH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in Oneโ€‘Shot๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2024RSSDROID: A Large-Scale Inโ€‘Theโ€‘Wild Robot Manipulation Dataset๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2024ICRAOpenโ€ฏXโ€‘Embodiment: Robotic Learning Datasets and RTโ€‘X Models๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2025RSSRoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2025IROSAgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2025arXivBRMData: Empowering Embodied Manipulation: A Bimanual-Mobile Robot Manipulation Dataset for Household Tasks๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ

Simulation Environments and Benchmarks

YearVenuePaperWebsiteCodeData
2022CoRLBEHAVIORโ€‘1K: A Humanโ€‘Centered, Embodied AI Benchmark with 1,000 Everyday Activities and Realistic Simulation๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2020CVPRALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2020RA-LRLBench: The Robot Learning Benchmark & Learning Environment๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2024arXivPerActยฒ: Benchmarking and Learning for Robotic Bimanual Manipulation Tasks๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2020CoRLMetaโ€‘World: A Benchmark and Evaluation for Multiโ€‘Task and Meta Reinforcement Learning๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2019CoRLRelay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2023NeurIPSLIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2022RA-LCALVIN: A Benchmark for Languageโ€‘Conditioned Policy Learning for Longโ€‘Horizon Robot Manipulation Tasks๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2024arXivMIKASA: Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learning๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2024CoRLSIMPLER: Evaluating Realโ€‘World Robot Manipulation Policies in Simulation๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2019ICCVHabitat: A Platform for Embodied AI Research๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2021NeurIPSHabitatโ€ฏ2.0: Training Home Assistants to Rearrange their Habitat๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2024ICLRHabitatโ€ฏ3.0: A Coโ€‘Habitat for Humans, Avatars and Robots๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2020CVPRSAPIEN: A Simulated Part-based Interactive Environment๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2024RSSTheโ€ฏColosseum: A Benchmark for Evaluating Generalization for Robotic Manipulation๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2025ICCVVLABench: A Largeโ€‘Scale Benchmark for Languageโ€‘Conditioned Robotics Manipulation with Longโ€‘Horizon Reasoning Tasks๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ

Human Behavior Datasets

YearVenuePaperWebsiteCodeData
2022CVPREgo4D: Around the World in 3,000 Hours of Egocentric Video๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2024CVPREgoโ€‘Exo4D: Understanding Skilled Human Activity from Firstโ€‘ and Thirdโ€‘Person Perspectives๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2024arXivEgoPlanโ€‘Bench: Benchmarking Egocentric Embodied Planning with Multimodal Large Language Models๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2024arXivEgoVidโ€‘5M: A Largeโ€‘Scale Videoโ€‘Action Dataset for Egocentric Video Generation๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2018ECCVScaling Egocentric Vision: The EPICโ€‘KITCHENS Dataset๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2024ECCVCOMโ€ฏKitchens: An Unedited Overheadโ€‘View Video Dataset as a Visionโ€‘Language Benchmark๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2019ICCVEgoVQA: An Egocentric Video Question Answering Benchmark Dataset๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2022NeurIPSEgoTaskQA: Understanding Human Tasks in Egocentric Videos๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2025arXivEgoDex: Learning Dexterous Manipulation from Largeโ€‘Scale Egocentric Video---
2024RSSDexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2024arXivEgoMimic: Scaling Imitation Learning via Egocentric Video๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2025CoRLHumanoid Policy ~ Human Policy๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2025arXivReal2Render2Real: Scaling Robot Data Without Dynamics Simulation or Robot Hardware๐ŸŒ๐Ÿ’ป-

Embodied Datasets and Benchmarks

YearVenuePaperWebsiteCodeData
2018CVPREQA: Embodied Question Answering๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2018CVPRIQA: Visual Question Answering in Interactive Environments-๐Ÿ’ป-
2019CVPRMTโ€‘EQA: Multiโ€‘Target Embodied Question Answering๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2019CVPREmbodied Question Answering in Photorealistic Environments with Point Cloud Perception๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2023ICLREQAโ€‘MX: Embodied Question Answering using Multimodal Expression---
2024CVPROpenEQA: Embodied Question Answering in the Era of Foundation Models๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ
2024ICLRLoTaโ€‘Bench: Benchmarking Languageโ€‘oriented Task Planners for Embodied Agents๐ŸŒ๐Ÿ’ป๐Ÿ“ฆ

Star History Chart

Citation

If you find this survey helpful for your research or applications, please consider citing it using the following BibTeX entry:

@misc{shao2025largevlmbasedvisionlanguageactionmodels,
      title={Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey}, 
      author={Rui Shao and Wei Li and Lingsen Zhang and Renshan Zhang and Zhiyang Liu and Ran Chen and Liqiang Nie},
      year={2025},
      eprint={2508.13073},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2508.13073}, 
}

Contact Us

For any questions or suggestions, please feel free to contact us at:

Email: shaorui@hit.edu.cn and liwei2024@stu.hit.edu.cn