World Pilot: Steering Vision-Language-Action Models with World-Action Priors

Zefu Lin, Rongxu Cui, Junjia Xu, Xiaojuan Jin, Wenling Li, Lue Fan, and Zhaoxiang Zhang

Overview   •   News   •   Documentation   •   Citation   •   Acknowledgements

Overview

World Pilot steers a VLA with priors from a World-Action Model. VLA methods generate actions from a VLM's encoding of the scene. World Pilot adds two priors from a WAM into the decision chain, with Latent Steering routing a scene-evolution latent into VLM hidden states and Action Steering feeding a trajectory-level motion prior to the action generator. This gives the VLA an anticipated view of the scene and a motion hint alongside its semantic conditioning. World Pilot reaches state-of-the-art performance on LIBERO-Plus and real-robot tasks.

✨ News

• [2026.6.9] WorldPilot model weights are now available on Hugging Face. Feel free to try them out! 🚀
• [2026.6.8] The WorldPilot codebase is released, including training and evaluation. Model weights are coming soon. 🚀
• [2026.6.7] WorldPilot is now live on arXiv. The code is coming soon. 🚀

📚 Documentation

The documentation below covers environment setup, training, and public evaluation. Start with the installation guide, then follow the training or evaluation notes for your workflow.

• Installation
• Training
• Evaluation

🤗 Model Zoo

We release our pretrained model parameters and precomputed LIBERO cache on Hugging Face.

• Model weights: Chedan86/WorldPilot-LIBERO
• Precomputed LIBERO cache: Chedan86/WorldPilot-LIBERO-precompute

📄 Citation

If WorldPilot helps your research, we would appreciate a citation using the BibTeX entry below.

@article{worldpilot2026,
  title={World Pilot: Steering Vision-Language-Action Models with World-Action Priors},
  author={Zefu Lin and Rongxu Cui and Junjia Xu and Xiaojuan Jin and Wenling Li and Lue Fan and Zhaoxiang Zhang},
  journal={arXiv preprint arXiv:2606.12403},
  year={2026}
}

🤝 Acknowledgements

We sincerely thank the teams behind ABot-Manipulation, cosmos-policy, LIBERO, LIBERO-plus, LeRobot for their outstanding work.