We introduce two complementary benchmarks built on the same high-fidelity 3D Gaussian Splatting (3DGS) reconstruction stack — ABotN-PointBench for coordinate-conditioned navigation and ABotN-POIBench for name-conditioned POI navigation — to advance evaluation of closed-loop, social-rule-aware visual navigation in real-world indoor and outdoor environments. We additionally release a re-curated Short-Horizon OVON variant that isolates the recognition-and-approach phase of object-goal navigation. All benchmarks and the evaluation toolkit are released as part of the ABot-N1 project.

Benchmarks

Benchmark	Task	Goal	Scenes	Episodes
ABotN-PointBench	Point-Goal	Navigate to (x, y) coordinates	31 real-world 3DGS scenes (16 indoor + 15 outdoor)	465
ABotN-POIBench	POI-Goal	Navigate to a named POI entrance	11 commercial areas, 126k m²	163 POIs
Short-Horizon OVON	Object-Goal	Find and approach a target object category	36 HM3D scenes (OVON Val-Unseen)	2,443

ABotN-PointBench and ABotN-POIBench are newly introduced benchmarks with full evaluation tooling in this repository. Short-Horizon OVON is a visibility-filtered variant of the HM3D-OVON benchmark; see its documentation for integration details.

📦 Datasets — Released

All benchmark data is publicly available on 🤗 HuggingFace and 🤖 ModelScope:

Dataset	Download
ABotN-PointBench	🤗 HuggingFace | 🤖 ModelScope
ABotN-POIBench	🤗 HuggingFace | 🤖 ModelScope
ABotN-Short-Horizon-OVON	🤗 HuggingFace | 🤖 ModelScope

See Getting Started for download and setup instructions.

Key features:

• Photorealistic multi-view RGB observations from real-world 3DGS scene reconstructions
• Closed-loop evaluation with collision-aware success metrics (SR_<3col outdoor, SR_<1col indoor)
• Social-rule-aware traversability scoring via annotated walkability maps
• Standardized evaluation protocols with fixed thresholds for reproducible comparison
• Minimal agent interface: implement reset() + predict() to evaluate any model

Benchmark Results

Point-Goal — ABotN-PointBench (Outdoor)

Method	SR<3col↑	SPL↑
GNM	39.1	36.7
ViNT	62.2	62.2
NoMaD	56.0	55.7
CityWalker	48.9	48.3
SocialNav	72.0	71.9
ABot-N1	92.9	91.4

Point-Goal — ABotN-PointBench (Indoor)

Method	SR<1col↑	SPL↑
GNM	26.7	26.6
ViNT	27.9	27.9
NoMaD	20.0	19.6
CityWalker	21.7	21.6
SocialNav	42.5	42.5
ABot-N1	95.4	93.7

POI-Goal — ABotN-POIBench

Method	SR<2m↑	SPL↑
ViNT	19.0	18.2
OmniNav (vanilla)	23.9	22.4
OmniNav (BridgeNav)	34.4	31.5
POINav	42.3	40.3
ABot-N1	77.3	72.6

Object-Goal — Short-Horizon OVON

Method	SR↑	SPL↑	DTG↓
StreamVLN	39.7	15.8	2.368
NaVILA	55.4	26.1	1.811
Uni-NaVid	68.7	34.5	1.495
ABot-N1	84.9	51.8	0.822

Full results and per-difficulty breakdowns available in the technical report.

Architecture

┌──────────────────────────┐         ┌──────────────────────────────┐
│  3DGS Render Server      │  HTTP   │  Evaluation Environment      │
│  Python 3.8, CUDA 11     │◄───────►│  pip install abotn-bench     │
│  render_server/          │         │  import abotn_evaluator      │
└──────────────────────────┘         └──────────────────────────────┘

Quick Start

# Install
git clone <repository_url>
cd <repository_name> && pip install -e .

# Deploy render server (separate conda env, CUDA 11 required)
conda env create -f render_server/environment.yml
conda activate abotn_render
bash scripts/start_PointGoal_outdoor_render_server.sh   # set SCENES_ROOT first

# Evaluate your agent
python -m abotn_evaluator.point_goal.runner \
    --agent-module your_agent:YourAgent \
    --data-dir /path/to/pointbench/outdoor/trajectory \
    --render-url http://localhost:7036/render_gs \
    --mode outdoor

Agent Interface

from abotn_evaluator.interface.point_goal import BasePointGoalAgent, Observation, WaypointPrediction

class YourAgent(BasePointGoalAgent):
    def reset(self): ...
    def predict(self, observation: Observation) -> WaypointPrediction:
        # observation.images: Dict[str, ndarray] — multi-view RGB (left/front/right)
        # observation.target_position: ndarray — [front, left] in metres
        # observation.distance_to_goal: float
        return WaypointPrediction(waypoint=..., arrive=...)

For POI-Goal, use BasePoiGoalAgent — the observation adds a poi_name: str field.

Metrics

Task	Metrics	Description
Point-Goal (Outdoor)	SR<3col, SPL	Success rate under a 3-collision budget; path efficiency vs. A* reference
Point-Goal (Indoor)	SR<1col, SPL	Success rate under strict zero-collision criterion
POI-Goal	SR<2m, SPL	Entrance arrival within 2 m; global and per-POI
Object-Goal (Short-Horizon OVON)	SR, SPL, DTG	Success rate, path efficiency, distance-to-goal at termination

Documentation

Getting Started	Installation, render server setup, data download
Point-Goal Evaluation	Outdoor/indoor protocol, evaluation commands
POI-Goal Evaluation	POI-Goal protocol, evaluation commands
Short-Horizon OVON	Visibility-filtered Object-Goal variant for Habitat-sim
API Reference	Observation/WaypointPrediction fields, coordinate system, CLI flags
Custom Agents	Coordinate adaptation, wrapper pattern

Citation

If you find ABotN-Bench useful in your research, please cite the technical report:

@misc{gong2026abotn1generalvisuallanguage,
      title={ABot-N1: Toward a General Visual Language Navigation Foundation Model}, 
      author={Ruiyan Gong and Yingnan Guo and Junjun Hu and Jintao Kong and Xiaoxu Leng and Tianlun Li and Weize Li and Fei Liu and Zhicheng Liu and Jia Lu and Minghua Luo and Chenlin Ming and Yanfen Shen and Jiyue Tao and Zhengbo Wang and Mingyang Yin and Minqi Gu and Zihao Guan and Wei Guo and Guoqing Liu and Huachong Pang and Menglin Yang and Zeqian Ye and Xiaoxiao Geng and Zhining Gu and Honglin Han and Di Jing and Hongyu Pan and Mingchao Sun and Kuan Yang and Jianfang Zhang and Yanghong Chen and Ye He and Wei Mei and Jiahao Shi and Xiangpo Yang and Yanqing Zhu and Yang Cai and Jingjing Ma and Shihui Su and Zixiao Tang and Linbo Zheng and Zedong Chu and Xiaolong Wu and Wenbin Tang and Mu Xu},
      year={2026},
      eprint={2607.10383},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.10383}, 
}

License

Apache-2.0. render_server/ retains the 3D Gaussian Splatting license.

Acknowledgments

ABotN-Bench is developed by AMAP CV Lab as part of the ABot-N1 project. We thank the open-source community for 3D Gaussian Splatting, Habitat, and HM3D-OVON.