Overview

wanderland-lab is a modular library for closed-loop navigation evaluation in photorealistic, geometrically grounded Wanderland scenes. You point it at a scene + a set of goal-reaching episodes, plug in a policy, and get standard navigation KPIs (success rate, SPL, navigation error, path length) back as JSON. The same env doubles as a Gym-compatible RL environment for residual fine-tuning on top of a frozen base policy.

Key features

• A policy-agnostic IsaacLab env. WanderlandEnv (a DirectRLEnv subclass) loads a Wanderland USD, spawns a dynamic agent + first-person camera, and exposes a batched observation dict. Vectorize with cfg.scene.num_envs=N.
• A simple Policy protocol. Bring any navigation model by implementing one act(obs) -> Action method — no env changes, no client/server split. CityWalker ships as the reference implementation.
• A Hydra benchmark CLI. Run N episodes against any policy across one or many scenes; results are written as per-scene KPI JSON in a stable schema.
• Residual RL fine-tuning. SB3 PPO on top of a frozen base policy via ResidualPolicyWrapper, with a multi-scene curriculum that chains checkpoints across scenes.

Table of contents

• Installation
• Getting the data and weights
• Walkthrough
  • 1. Benchmark a single scene
  • 2. Read the results
  • 3. Sweep multiple scenes
  • 4. Write your own policy
  • 5. Residual RL fine-tuning
• Repository layout
• Documentation
• Contributing
• Citation
• License
• Acknowledgements

Installation

The whole Isaac Sim 5.1 + IsaacLab 2.3 stack is managed by uv. One uv sync resolves isaacsim, isaaclab, the +cu128 PyTorch build, and wanderland-lab itself.

Requirements: Linux x86_64, an NVIDIA GPU (≥24 GB VRAM recommended), and a CUDA 12.8-compatible driver.

git clone https://github.com/ai4ce/wanderland-lab.git
cd wanderland-lab
uv sync

Getting the data and weights

Dataset. Wanderland scenes are on HuggingFace Hub at ai4ce/wanderland. Use scripts/download.py to fetch them straight into the layout the benchmark expects (data/wanderland/<scene_id>/{scene.usdz,episodes.json}). Scene selection goes through the public manifest, which is the source of truth for currently released scenes and their quality_tier (showcase / evaluation_ready / training_ready):

# A whole benchmark scene list...
uv run python scripts/download.py --scene-list data/wanderland/scene_list/sanity_v0.0.1.txt

# ...or a quality tier, specific scenes, or the first N of all manifest scenes.
uv run python scripts/download.py --quality-tier showcase
uv run python scripts/download.py --scenes <scene_id> [<scene_id> ...]
uv run python scripts/download.py --count 5

By default this pulls only the navigation modality (scene.usdz + episodes.json) — the two files this benchmark uses. Pass --modality {3d,nvs,full} for the COLMAP / point-cloud / image-tarball data (large; not consumed here), --output <dir> to place data elsewhere, --release-version {v1,v2} to filter by release profile, or --all for every manifest scene. Add --dry-run to preview a selection's files before fetching. Run --help for the full set of options.

(Or set WANDERLAND_DATA=/abs/path/to/wanderland.) See data/wanderland/README.md for the full layout. Starter scene lists live in data/wanderland/scene_list/.

CityWalker weights. Nothing to download by hand — the model and its DINOv2 backbone (facebook/dinov2-base) auto-download from HF Hub on first run (~833 MB). The model code travels with the ai4ce/citywalker Hub repo via auto_map + trust_remote_code=True, so you don't even need this package installed to load it.

Walkthrough

A zero-to-results tour: benchmark a scene, read the metrics, sweep, then extend with your own policy and residual RL. All sim commands assume the uv-native install above (prefix with OMNI_KIT_ACCEPT_EULA=YES to accept NVIDIA's EULA non-interactively on first boot).

1. Benchmark a single scene

SCENE_ID=<scene_id>
DATA=$(pwd)/data/wanderland          # absolute — Hydra changes the working dir

OMNI_KIT_ACCEPT_EULA=YES uv run wanderland-lab benchmark \
    policy=citywalker \
    scene_id=${SCENE_ID} \
    env.usd_path=${DATA}/${SCENE_ID}/scene.usdz \
    env.episodes_json=${DATA}/${SCENE_ID}/episodes.json \
    runner.query_every_n=5 \
    output.dir=$(pwd)/outputs/citywalker

The wanderland-lab benchmark console script wraps the Hydra CLI (python -m wanderland_lab.benchmark.cli) and sets the LD_PRELOAD=libcarb.so bootstrap Isaac Sim needs. Anything after the verb is a standard Hydra key=value override. Pass absolute paths for env.usd_path / env.episodes_json / output.dir — Hydra switches the process to a per-run working directory, so relative paths won't resolve.

2. Read the results

Results are written per scene, in a stable schema:

outputs/citywalker/<scene_id>/<scene_id>_benchmark_results.json

Each file holds a summary block plus per-episode records (schema_version=1):

{
  "scene_id": "...",
  "schema_version": 1,
  "policy": "citywalker",
  "summary": {
    "num_episodes": 12,
    "num_success": 8,
    "success_rate": 0.667,
    "spl_euclidean_mean": 0.41,
    "spl_geodesic_mean": 0.39,         // null if no episode had a geodesic distance
    "navigation_error_mean": 1.2
  },
  "episodes": [
    {
      "episode_id": 5,
      "success": true,
      "termination": "goal_reached",   // or "stuck", "frame_cap", ...
      "num_steps": 120,
      "metrics": { "path_length": 12.3, "spl_geodesic": 0.66, ... },
      "geodesic_distance": 11.5        // echoed from episodes.json, may be null
    }
    // ...
  ]
}

Metric definitions (SPL euclidean + geodesic, navigation error, path length) live in src/wanderland_lab/metrics/navigation.py.

3. Sweep multiple scenes

A small loop over a scene list is all you need:

SCENE_LIST=data/wanderland/scene_list/sanity_v0.0.1.txt
DATA=$(pwd)/data/wanderland
OUT=$(pwd)/outputs/citywalker_sweep

while read -r scene_id; do
    [[ -z "$scene_id" || "$scene_id" =~ ^# ]] && continue   # skip blanks + comments
    OMNI_KIT_ACCEPT_EULA=YES uv run wanderland-lab benchmark \
        policy=citywalker scene_id=${scene_id} \
        env.usd_path=${DATA}/${scene_id}/scene.usdz \
        env.episodes_json=${DATA}/${scene_id}/episodes.json \
        runner.query_every_n=5 \
        output.dir=${OUT}
done < ${SCENE_LIST}

Each invocation spawns its own SimulationApp — IsaacLab assumes one SimulationApp lifetime per scene, so don't combine scenes in a single process.

4. Write your own policy

The only extension point is the Policy protocol. A policy is anything that maps an observation dict to a body-frame Action:

from wanderland_lab.policies.base import Action, Policy, PolicyCapabilities

class MyPolicy(Policy):
    @property
    def capabilities(self) -> PolicyCapabilities:
        # context_size = how many past RGB frames the env should buffer & pass.
        return PolicyCapabilities(context_size=1)

    def reset(self, session_id: str) -> None:
        ...  # clear per-episode state (history buffers, RNGs)

    def act(self, observation) -> Action:
        # observation has: images (T,H,W,3), ego_pos, ego_yaw, goal, dt, ...
        # Return body-frame velocities; the env handles body->world.
        return Action(vx=0.5, vy=0.0, yaw_rate=0.0)

    def act_batch(self, observations):
        return [self.act(o) for o in observations]   # override for real GPU batching

Drop a Hydra config at configs/benchmark/policy/<name>.yaml pointing _target_ at your class, then run it with policy=<name>. The full worked example (capabilities, the goal-image path, batched inference, Hydra wiring) is in docs/adding_a_policy.md.

5. Residual RL fine-tuning

Train a PPO residual on top of a frozen base policy. Single scene:

DATA=$(pwd)/data/wanderland

OMNI_KIT_ACCEPT_EULA=YES uv run python -m wanderland_lab.rl.train \
    policy=citywalker scene_id=${SCENE_ID} \
    env.usd_path=${DATA}/${SCENE_ID}/scene.usdz \
    env.episodes_json=${DATA}/${SCENE_ID}/episodes.json \
    train.num_envs=8 train.n_timesteps=500000 \
    train.save_dir=$(pwd)/outputs/sb3

Multi-scene curriculum (one fresh SimulationApp per scene, checkpoints chained via train.resume_from):

scripts/train_curriculum.sh \
    --scene-list data/wanderland/scene_list/sanity_v0.0.1.txt \
    --base-out outputs/sb3_curriculum \
    --per-scene-timesteps 500000 --num-envs 8 --policy citywalker

Pass --gpu k to fan out across GPUs; --dry-run previews the per-scene invocation. Checkpoints land at ${train.save_dir}/<policy>_<timestamp>/model.zip.

Repository layout

src/wanderland_lab/
├── envs/              WanderlandEnv (DirectRLEnv) + scene + coords + episode loader
├── policies/          Policy protocol, RandomPolicy, CityWalkerPolicy
├── models/citywalker/ HF AutoModel port of CityWalker (publishes to ai4ce/citywalker)
├── wrappers/          ResidualPolicyWrapper + obs-vector helpers
├── rl/                train.py + tasks/ (gym registration)
├── benchmark/         cli.py (hydra), runner.py, logger.py
├── datasets/          episodes.py, scene_list.py
└── metrics/           navigation.py (SPL, navigation error, path length)

configs/
├── benchmark/         default + env + policy/{citywalker,random}
└── train/             default + env + wrapper/residual + policy/{citywalker,random}

scripts/
├── download.py           # fetch Wanderland scenes from HF Hub into data/wanderland/
└── train_curriculum.sh   # multi-scene PPO curriculum (one process per scene)

A few conventions worth knowing if you contribute: the env stays policy-agnostic (composition with a base policy lives in wrappers/); policies emit body-frame (vx, vy, yaw_rate) and the env's _apply_action is the only body→world conversion site; override act_batch for hot paths.

Documentation

Topic	File
Adding a new policy	docs/adding_a_policy.md
Coordinate convention	docs/coordinate_frames.md
Reproducing baselines	docs/reproducing_baselines.md

Contributing

The extension model is the Policy protocol — add a navigation model by writing a Policy subclass + a Hydra config, no env changes required. docs/adding_a_policy.md is a complete worked example. Issues and pull requests welcome.

Citation

If you use wanderland-lab, please cite the Wanderland paper:

@inproceedings{liu2026wanderland,
  title     = {Wanderland: Geometrically Grounded Simulation for Open-World Embodied AI},
  author    = {Liu, Xinhao and Li, Jiaqi and Deng, Youming and Chen, Ruxin and Zhang, Yingjia and Ma, Yifei and Guo, Li and Li, Yiming and Zhang, Jing and Feng, Chen},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages     = {1041--1052},
  year      = {2026}
}

License

Licensed under the Apache License 2.0.

Acknowledgements

Built on IsaacLab / Isaac Sim. The CityWalker baseline is a port of CityWalker; the Wanderland scenes come from the Wanderland project.