In-N-On: Scaling Egocentric Manipulation with in-the-wild and on-task Data

Website | arXiv | Data | Model

Introduction

This repository works in a few simple steps.

1. Converts diverse egocentric datasets to human-centric representation.
2. Train a policy that imitaties human behavior.
3. The policy predicts future human movement, which can be retargeted and deployed on actual bimanual egocentric robots.

We support pre-training and post-training on mixed robot and human data.