README.md

A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards

[Project Page] [Paper]

Shivansh Patel^1*, Xinchen Yin^1*, Wenlong Huang², Shubham Garg³, Hooshang Nayyeri³, Li Fei-Fei², Svetlana Lazebnik¹, Yunzhu Li^4* (* indicates equal contributions)

¹University of Illinois at Urbana-Champaign, ²Stanford University, ³Amazon, ⁴Columbia University

This is the official demo code for IKER. Currently, it includes RL training code for the Shoe Place task, with additional code to be released soon. This implementation is based on Issac Gym. IKER samples keypoints from the scene and utilizes VLMs to generate Python-based reward functions conditioned on these keypoints. We reconstruct real-world scenes in simulation and use the generated rewards to train reinforcement learning policies, which are then deployed into the real world—forming a real-to-sim-to-real loop.

Setup Instructions

1. Install the repository dependencies by running:

conda env create -f environment.yml

2. Download the Isaac Gym Preview 4 release from the website, and install it in the same conda environment. Follow the installation instructions provided in the documentation.

3. Verify that Isaac Gym is working by running one of Isaac Gym's examples. If you encounter any issues, refer to the Isaac Gym Preview 4 troubleshooting guide.

4. Run the following line:

pip install -e .

This section has been adapted from the original Isaac Gym documentation. Please refer to their pages for further troubleshooting.

Run RL Training

To run RL training part for IKER, run:

python isaacgymenvs/train.py task=ShoePlace