Getting Started - Franka DROID

March 27, 2026 ยท View on GitHub

Software installation

On your NUC

If you already have the DROID framework installed, skip this step! Otherwise, follow these instructions. If you wish, you can install only polymetis instead of DROID.

On your Inference PC

First, install the ZED SDK, matching your CUDA version. Then:

# Create fresh conda env for polymetis
conda create -n molmobot python=3.8
conda activate molmobot
conda install -c pytorch -c fair-robotics -c aihabitat -c conda-forge polymetis

# Install ZED SDK python bindings
cd /usr/local/zed
python get_python_api.py

# Install dependencies
pip install git+https://github.com/allenai/ai2_robot_infra.git#egg=ai2_robot_infra[zed]

Policy Evaluation

Hardware Setup

Modify the eval config to match your hardware setup. In particular, edit:

  • robot.robot_host to be the IP of the NUC
  • robot.cameras.wrist_camera.id to be the serial number of your wrist camera (ZED Mini)
  • robot.cameras.exo_camera_1.id to be the serial number of your exo camera (ZED 2/2i)

If you wish, you can also enable W&B support (upload rollout videos by filling out the wandb block of the config and setting wandb.enabled to true).

Policy Setup

Each MolmoBot policy class maintains its own python package, so you can install and use what you need separately. To install and run the policy server for each policy class, see the corresponding documentation:

Running your policy

  1. Log into Franka Desk to unlock joints and enable FCI.

  2. Start the robot and gripper servers on the NUC (see polymetis docs). For example, if DROID is installed on the NUC:

    # In terminal 1
ssh droid  # ssh into the NUC
conda activate polymetis-local
cd droid/droid/fairo/polymetis
launch_robot.py robot_client=franka_hardware

    # In terminal 2
ssh droid  # ssh into the NUC
conda activate polymetis-local
cd droid/droid/fairo/polymetis
launch_gripper.py gripper=robotiq_2f

  3. Start up your policy server in terminal 3. See policy documentation for more information. For example, to run MolmoBot-DROID:

    # In terminal 3
cd MolmoBot/MolmoBot
source .venv/bin/activate
PYTHONPATH=. python launch_scripts/serve_molmo.py --hf-repo allenai/MolmoBot-DROID --action-type joint_pos

  4. Activate the environment and control the robot. When running the script, the robot will first go to a home position, and then prompt the user to press enter to begin the episode.

    # In terminal 4
conda activate molmobot
python scripts/droid/run_policy.py task="put the red mug in the black bowl"

    NOTE: if you get ffmpeg/ImageIO errors, you may need to run conda remove --force ffmpeg and install it system-wide with sudo apt install ffmpeg.