Agent Development Guide

June 29, 2026 ยท View on GitHub

This file is the entry point for agentic coding agents operating in this repository. Start here, then follow the links below for details on each subsystem.

Reasoning rule: Agents must follow .agent/FIRST_PRINCIPLES.md โ€” derive every action from evidence in this repo and from explicit contracts; never guess or pattern-match.


๐Ÿš€ Agent boot sequence

When you pick up a task in this repository, read in this order. Stop as soon as the question is answered; don't burn context on files that aren't relevant.

  1. This file (AGENTS.md) โ€” the dispatch layer. Tells you which subsystem owns the task.
  2. source/AGENTS.md โ€” repository module map (one line per geniesim_* peer). Picks the package you'll be editing.
  3. source/<pkg>/AGENTS.md โ€” canonical guide for that one package. File layout, command surface, architectural rules, troubleshooting. This is the source of truth for "how does X work".
  4. source/<pkg>/skills/<name>/SKILL.md โ€” concrete recipe for a common workflow (run a benchmark, launch a scene, add a robot, โ€ฆ). Self-contained: prerequisites, copy-paste commands, gotchas.
  5. Source files referenced by 1โ€“4. Only after the doc trail above narrows the scope.

Side trail for repo-wide architecture (rarely needed):

  • .agent/ โ€” thin redirects to the per-package AGENTS.md of the most-asked-about peers. Treat as a dispatcher; the per-package file is canonical.

Anti-pattern: starting with grep / find before reading source/AGENTS.md. The package table tells you which directory to scope a search to.


๐Ÿงž Canonical commands

The geniesim CLI is the single entry point for every operator-level workflow. Prefer these over ad-hoc invocations; they handle path resolution, interpreter selection, and environment normalisation.

VerbWhat it does
geniesim bootstrapInstall every peer distribution in topological order (the umbrella)
geniesim statusPer-distribution health probe โ€” never raises on missing siblings
geniesim doctorDiagnose & repair (status + rosdep + env)
geniesim docker {build,up,into,down,logs}Manage the Genie Sim container (Isaac Sim 5.1 / 6.0 / 4.5)
geniesim ros build {dev,release,cleanup}colcon build the ROS 2 workspace
geniesim ros doctorRepair rosdep
geniesim benchmark run <CONFIG>Run one benchmark task
geniesim benchmark check-inferenceProbe an inference WebSocket server
geniesim teleop runLaunch the VR / Pico teleop loop
geniesim deploy [MODULE]Build pure-Python wheel(s) into ./deploy/
geniesim version / geniesim env / geniesim completion bash|zshOperator utilities

Full surface: source/geniesim_cli/AGENTS.md.

Never suggest pip install geniesim or pip install geniesim_assets โ€” those distributions are not on PyPI. Always redirect through geniesim bootstrap.


Core References

DocumentPurpose
.agent/FIRST_PRINCIPLES.mdEvidence-first reasoning rules โ€” derive every action from repo facts, never guess
source/AGENTS.mdRepository module map โ€” one row per geniesim_* peer with its docs + skills
source/README.mdGitHub-facing module index โ€” same map, reader-oriented

.agent/ is a thin dispatch layer for cross-cutting peers; each redirects to its canonical source/<pkg>/AGENTS.md. Leaf peers go straight to source.

PeerGuide
geniesim (umbrella).agent/geniesim.md
geniesim_cli.agent/geniesim_cli.md
geniesim_benchmark.agent/geniesim_benchmark.md
geniesim_generator.agent/geniesim_generator.md
geniesim_ros.agent/geniesim_ros.md
geniesim_teleop.agent/geniesim_teleop.md
geniesim_worldsource/geniesim_world/AGENTS.md
data_collectionsource/data_collection/AGENTS.md
rlinf_geniesimsource/rlinf_geniesim/README.md
scene_reconstructionsource/scene_reconstruction/README.md

Repository Layout (quick map)

source/
โ”œโ”€โ”€ geniesim/             umbrella meta-package (no code, only deps)
โ”œโ”€โ”€ geniesim_cli/         CLI dispatcher; owns the `geniesim` console script
โ”œโ”€โ”€ geniesim_benchmark/   benchmark tasks, scoring, LLM eval configs
โ”œโ”€โ”€ geniesim_generator/   scene generation, procedural layout
โ”œโ”€โ”€ geniesim_ros/         Genie Sim RT Engine โ€” ROS 2 workspace
โ”œโ”€โ”€ geniesim_teleop/      VR / Pico teleoperation bridge
โ”œโ”€โ”€ geniesim_world/       multimodal spatial world model (pano โ†’ 3D)
โ”‚
โ”‚   โ”€โ”€ separately-maintained (not `geniesim_*` peers) โ”€โ”€
โ”œโ”€โ”€ data_collection/      data collection client/server
โ”œโ”€โ”€ rlinf_geniesim/       RL training (RLinf, human-in-the-loop)
โ”œโ”€โ”€ scene_reconstruction/ 3D reconstruction pipeline
โ””โ”€โ”€ external/             vendored third-party code

See source/AGENTS.md for the full package table.


Do Not

  • Generate or guess URLs โ€” only use URLs present in local files.
  • Hardcode absolute paths outside the repo root.
  • Skip geniesim_cli._style colors when writing CLI output.
  • Put ROS-only deps (rclpy, cv-bridge, โ€ฆ) in the core geniesim or geniesim_cli dependencies.
  • Suggest pip install geniesim or pip install geniesim_assets โ€” not on PyPI; redirect to geniesim bootstrap.
  • Duplicate .agent/*.md content from the per-package AGENTS.md it points at. The .agent/ files are dispatchers, not canonical sources.