RunPod / vast.ai helper scripts

Scripts for running the pipeline on a rented GPU box (RunPod, vast.ai, anything you can SSH into). They handle:

• syncing code + selected source views up to the remote
• bootstrapping the env
• pulling results back down

All scripts live in runpod/ and resolve paths relative to the repo root, so you can run them from anywhere.

SSH setup

Both send.sh and retrieve.sh connect via:

ssh -p $SSH_PORT -i $SSH_KEY root@$SERVER

Common env vars

var	default	notes
SERVER	(required)	host/IP of the GPU box
SSH_PORT	27406
SSH_KEY	~/.ssh/id_ed25519
REMOTE_DIR	workspace/ri3d	remote project root (relative to ~)

send.sh — local → server

Pushes the codebase and (optionally) a deterministic subset of source images plus arbitrary extra paths.

[SCENE=bonsai] [N_VIEWS=3] ./runpod/send.sh [-f] [extra_path ...]

What gets sent, in order:

1. Codebase. rsync with --filter=':- .gitignore' --exclude='.git/'. Anything gitignored (notably dataset/, output/, the venv) is skipped. mtime-based update — changed files always sync.
2. Selected source views, only if SCENE is set. Mirrors select_views() in src/step1_dust3r.py:54, reimplemented in stdlib-only Python inside the script (so we don't pull in torch just to pick filenames). Selection priority:
   1. dataset/<SCENE>/views.txt if present (also shipped, so the server reproduces the same selection)
   2. N_VIEWS as comma-separated filenames (N_VIEWS=DSCF5566,DSCF5640,...)
   3. N_VIEWS as int → evenly spaced indices over the sorted image list
3. Extra positional paths. Each is rsynced from <repo>/<path> to workspace/ri3d/<path>, preserving structure. Works on files and dirs.
   • default: --ignore-existing (won't clobber remote files)
   • -f: overwrite remote with local

Examples:

SERVER=1.2.3.4 SCENE=bonsai ./runpod/send.sh                         # code + 3 evenly-spaced bonsai views
SERVER=1.2.3.4 SCENE=bonsai N_VIEWS=5 ./runpod/send.sh               # code + 5 views
SERVER=1.2.3.4 SCENE=bonsai N_VIEWS=DSCF5566,DSCF5800,DSCF5856 ./runpod/send.sh
SERVER=1.2.3.4 ./runpod/send.sh output/bonsai/stage1_renders         # code + extra (no overwrite)
SERVER=1.2.3.4 ./runpod/send.sh -f utils/cache.json output/bonsai    # code + extras, overwrite

If you keep a views.txt in the scene dir, it wins over N_VIEWS so the client and server always agree on which views the pipeline runs on.

retrieve.sh — server → local

Pulls selected subfolders/files off the server into <repo>/../output/.

SERVER=1.2.3.4 ./runpod/retrieve.sh <subpath> [<subpath> ...]

Each subpath is a path relative to the project root on both ends:

• remote source: workspace/ri3d/<subpath>/
• local dest: <repo>/../output/<subpath>/

The server's base segment (e.g. output/) is preserved in the local layout — that's why pulling output/bonsai/stage1_renders lands at ../output/output/bonsai/stage1_renders/.

Behavior:

• No --ignore-existing — remote changes overwrite local. (Identical files are still skipped by rsync's default size+mtime check.)
• One arg failing (typo, missing on remote) does not stop the others; the script exits non-zero overall if any failed.

Example:

SERVER=1.2.3.4 ./runpod/retrieve.sh output/bonsai/stage1_renders output/bonsai/stage2_renders

env-set.sh — bootstrap caches on the remote

Run once per fresh GPU box. Points pip and HuggingFace caches at ~/workspace/ (the persistent network volume on RunPod, so caches survive container restarts) and pins TORCH_CUDA_ARCH_LIST for gsplat's CUDA kernel build.

Source it, don't execute it — the exports only stick if the script runs in your current shell:

source ./runpod/env-set.sh

do.sh — one-shot bootstrap + train

End-to-end recipe meant to run on the remote after send.sh:

./runpod/do.sh

It sources env-set.sh, installs DUSt3R, installs requirements.txt, then runs prep + LOO + repair training on dataset/bonsai/. Edit it to point at a different scene or to add --train_inpaint / --optimize stages.

Typical workflow

# local
SERVER=1.2.3.4 SCENE=bonsai ./runpod/send.sh

# on the remote (over ssh)
cd ~/workspace/ri3d && ./runpod/do.sh

# back local, pull outputs
SERVER=1.2.3.4 ./runpod/retrieve.sh output/bonsai