Visual Planning

May 15, 2025 · View on GitHub

After efficient adaptation, AdaWorld can be controlled directly by raw action inputs, which enables visual planning. Please follow the setup of iVideoGPT to compile our world model and perform robot planning tasks on the VP2 benchmark.

Tip

Use Hugging Face to download the VP2 dataset. It will be much faster than the older links.

For Robosuite, you only require 5k_slice_rendered_256.hdf5.

For RoboDesk, you can use the scripted policies to generate the training trajectories locally if the download feels too slow.

You may need to batchify the sampling, reduce the denoising steps (to 3), and turn off classifier-free guidance (use IdentityGuider), otherwise it could take days.

<= Previous: [World Model Adaptation]

=> Next: [Trouble Shooting]