droid-franka-client

Thin DROID / Franka FR3 eval client. It controls the robot (via the DROID stack on the NUC) and queries a remote galaxea-fm policy server over a websocket+msgpack protocol (PROTOCOL.md). It depends on the policy server only through that protocol — no shared Python code.

NUC (robot control, docker)  ──zerorpc──►  Franka FR3
        ▲ zerorpc
Laptop (this client, docker) ──ws:8000──►  galaxea-fm policy server (GPU host)
   + ZED cameras            ◄──actions───

What's in here

droid/          # git submodule → galaxea fork of droid-dataset/droid (Franka/ZED/docker patches)
                #   docker assets live in droid/.docker/{nuc,laptop}
openpi_client/  # vendored (flattened) openpi client; eval only uses image_tools
eval/           # main.py / main_ui.py — the actual eval loop + GalaxeaPolicyClient
docker/         # Dockerfile.eval + compose: BUILD the eval env from scratch (see below)
  requirements.lock.txt   # ground-truth pinned deps captured from the working env
start_client.sh # docker exec the eval loop in the eval container
sync_main.sh    # copy eval scripts into the running eval container
.env.example    # all machine-specific config (copy to .env)
PROTOCOL.md     # the client↔server contract
LICENSE-G0.5    # license for Galaxea-maintained client code/docs
NOTICE          # attribution and third-party component notes
CHANGES.md      # public-release change notes

Reproducible environment. The eval runs in a py3.10 conda env (droid_policy: droid + pyzed + openpi_client + msgpack/websockets/tyro/torch). That env is built from docker/Dockerfile.eval — it is NOT a hand-committed image. Anyone who clones this repo can rebuild it with docker build, which is the whole point of the split.

Open-Source Notes

This repository is released under the G0.5 Community License Agreement (Non-Commercial + Limited Patent License). See LICENSE-G0.5 and NOTICE.

Component boundaries:

• droid/ is a Git submodule pinned to the public Galaxea DROID fork at https://github.com/OpenGalaxea/droid.git. See the submodule's own README, CHANGES.md, and NOTICE.
• openpi_client/ is a vendored, flattened subset of the OpenPI client package from Physical Intelligence. See licenses/LICENSE-OPENPI.
• docker/vendor/pyzed-4.0-cp310-cp310-linux_x86_64.whl is intentionally retained for reproducible Docker builds in restricted network environments. The wheel includes its own MIT license metadata; users must also comply with applicable Stereolabs ZED SDK / PyZED terms.

---

From-scratch setup (new machine)

Everything below assumes you cloned this repo and have Docker + the robot hardware (Franka FR3, NUC, two ZED cameras) wired up.

1. Clone with submodules

git clone git@github.com:OpenGalaxea/droid-franka-client.git && cd droid-franka-client
git submodule update --init --recursive   # pulls droid + its fairo/oculus_reader submodules

The droid fork reads all machine config from environment variables, so you never edit files inside the submodule.

2. Configure your machine

cp .env.example .env
vim .env        # set ROOT_DIR=<abs path to ./droid>, ROBOT_IP, NUC_IP, camera serials,
                # SUDO_PASSWORD, POLICY_HOST/PORT, EVAL_OUTPUT_DIR ...

That's all — do not source it by hand. .env is plain KEY=value, so the same file feeds three consumers automatically:

consumer	how it reads .env
docker compose (build/up)	auto-loads .env; we also pass --env-file ./.env explicitly below
start_client.sh / sync_main.sh	source it themselves, and inject it into the container via docker exec --env-file
parameters.py inside the container	gets the values because the container is launched / exec'd with them

Why no host source: main.py and parameters.py run inside the docker container; exporting vars in your host shell never reaches the container. Config must be injected at docker compose up / docker exec time — which the scripts do for you. The droid fork reads every machine value from os.environ, so you never edit files inside the submodule.

3. Build & launch the NUC (robot control)

The NUC docker is owned by the droid fork (droid/.docker/nuc/, comes in via the submodule). Run these on the NUC machine.

# build the image AND start the container (build alone won't create it):
docker compose --env-file ./.env -f droid/.docker/nuc/docker-compose-nuc.yaml up -d --build

# start the control services inside the running NUC container:
docker exec -d $NUC_CONTAINER bash /app/scripts/server/launch_server.sh    # zerorpc :4242
docker exec -d $NUC_CONTAINER bash /app/droid/franka/launch_robot.sh       # robot   :50051
docker exec -d $NUC_CONTAINER bash /app/droid/franka/launch_gripper.sh     # gripper :50052
sleep 5 && docker exec -d $NUC_CONTAINER bash /app/droid/franka/launch_gripper.sh  # 1st gripper launch usually fails

4. Build & launch the laptop eval container

The eval container is built from docker/Dockerfile.eval (py3.10 env with droid + pyzed + openpi_client + the pinned deps). The first build needs the upstream droid laptop base image (ZED SDK + CUDA); pull it or build it from the submodule:

# (once) base image — pulls from ghcr, or build from the droid submodule:
docker pull ghcr.io/droid-dataset/droid_laptop:fr3 \
  || docker compose --env-file ./.env -f droid/.docker/laptop/docker-compose-laptop.yaml build

# build + start the reproducible eval container
docker compose --env-file ./.env -f docker/docker-compose.eval.yaml up -d --build

# verify cameras
lsusb | grep ZED             # expect 4 lines (ZED-M wrist + ZED-2i external)

After a power cycle / USB re-plug, fix the ZED video device nodes (else the SDK won't see the cameras):

for dev in /dev/video*; do
  maj=$(( 16#$(stat -c '%t' "$dev") )); min=$(( 16#$(stat -c '%T' "$dev") ))
  docker exec $LAPTOP_CONTAINER bash -c "rm -f $dev && mknod $dev c $maj $min && chmod 666 $dev"
done

5. Start the policy server

Done in the server repo (galaxeafm). It must listen on $POLICY_PORT and output joint_position. See that repo's experiments/droid/start_server.sh.

6. Run eval

bash sync_main.sh          # push eval scripts into the laptop container (once / after edits)
bash start_client.sh       # runs eval/main.py against $POLICY_HOST:$POLICY_PORT
# main_ui.py additionally shows subtask + bbox overlays

Interaction: type the task instruction once → robot executes (Ctrl+C aborts a round) → after each round enter y (success) / n (fail) / q (quit). Stats print each round; after 10 rounds it auto-resets and exits. Output → /eval_data (mounted to $EVAL_OUTPUT_DIR).

Manual reset:

docker exec -it $LAPTOP_CONTAINER bash -c \
  "/root/miniconda3/envs/droid_policy/bin/python -c \
  'from droid.robot_env import RobotEnv; RobotEnv().reset()'"

---

Troubleshooting

Franka power-on: open https://$ROBOT_IP in Firefox (Chrome blocks the self-signed cert) → if E-stop was pressed, twist to release & clear errors → click the lock icon to release brakes (arm twitches, light turns blue) → Activate FCI.

grpc Robot context not valid / Net Exception: Firefox → Deactivate FCI → wait 2s → Activate FCI → restart launch_robot.sh.

Gripper receives signal but doesn't move: kill main.py in the laptop container, then on the NUC re-run launch_gripper.sh a few times to reset gripper state; retry main.py (a transient grpc error on first retry is normal).

---

Relationship to the policy server

The server (galaxeafm, repo foundationmodel/galaxeafm) is a separate codebase. This client knows nothing about it beyond PROTOCOL.md. Swapping policies requires zero changes here as long as the server honors that contract.