Jev robot control

Jev vs GPT-6 Astra vs GPT-4.1 mini: direct Cartesian control of an xArm7 in MuJoCo.

One apple. One plate. Each controller chooses an intent, then X/Y/Z movement directions and a gripper command. The shared executor applies one small motion increment and returns physical feedback. This repository includes the code, original recorded responses and trajectories, an offline verifier, and a synchronized three-column replay.

Watch/download the comparison

Recorded result

Controller	Outcome	Cycles	API calls	API cost	Wall time	Simulation time
Jev 1.13	Placed	113	226	$0.018825	181.847 s	36.16 s
GPT-6 Astra, low reasoning	Placed	106	212	$5.933624	707.274 s	33.92 s
GPT-4.1 mini	160-cycle limit reached	160	320	$0.288512	704.253 s	51.20 s

These are one seed-0 trial per controller, not success-rate estimates. The Jev/GPT-6 recordings are the latest paired run; mini is from the earlier paired run with the same initial scene, physics, action instructions and cycle budget. See the protocol and source IDs.

In these recordings, Jev's API cost was about 1/315 of GPT-6's and its wall time about 26%; GPT-6 used seven fewer control cycles. Model availability, provider pricing, network timing and fresh responses can change future results.

1. Replay the published recordings

Only Python 3.12 and the files in this repository are needed for the recorded web replay. No API key or GPU is required:

git clone https://github.com/openroboto-ai/jev-robot-control.git
cd jev-robot-control
python incremental_triple_app.py --port 8773

Open http://127.0.0.1:8773/. Click Play or Replay from start. The page aligns simulation time and holds each completed controller's actual final frame. Costs and wall times update from recorded responses. API waiting is excluded from playback and included in the wall-time metric. The server binds to loopback and offers read-only replay; it makes no model calls.

2. Verify the physical trajectories offline

Create a Python 3.12 virtual environment and install the pinned dependencies:

python -m venv .venv
# Linux/macOS:
source .venv/bin/activate
# Windows PowerShell: .venv\Scripts\Activate.ps1
python -m pip install -r requirements.txt
python check_manifest.py
python -m unittest discover -s . -p test_incremental.py -v
python verify_replay.py --output runs/replay-verification.json

The verifier checks archived source hashes, every recorded question and observation, command validation, costs and counts. It re-executes each recorded motor command in MuJoCo and compares every saved joint/fruit state (qpos) with a 1e-7 tolerance. It also checks the physical success result. No API calls, renderer, display server or learned weights are needed.

The reference run used Python 3.12 on Windows with MuJoCo 3.13.0. Dependency versions are pinned in requirements.txt. Cross-platform floating-point behavior may differ; the verifier reports differences rather than hiding them.

3. Run new model trials

New trials call OpenRouter and incur provider charges. Put your own key in the process environment or copy .env.example to .env and edit it locally. Never commit a key. Access to the listed models, including the experimental Jev decisions endpoint, is required. A model or endpoint error ends that trial; there is no silent fallback to another model.

# One controller; no OpenGL renderer required:
python reproduce.py --controller jev --seed 0 --no-render
python reproduce.py --controller gpt6 --seed 0 --no-render
python reproduce.py --controller mini --seed 0 --no-render

# All three, sequentially, including new MP4 recordings:
python reproduce.py --controller all --seed 0

Each independent trial starts from the same scene. Fresh outputs go into a new timestamped directory under runs/; the published examples stay unchanged. Use --max-cycles or --output-dir for an explicit budget or output location. The three-controller convenience command is sequential, so its wall timings are not a reconstruction of the original concurrent paired execution.

For the original two-controller live UI and concurrent independent simulations:

python incremental_compare_app.py --port 8774 --env-file .env --llm-model openai/gpt-6-astra --reasoning-effort low --pair-root runs/gpt6-pairs

Open http://127.0.0.1:8774/ and click Run comparison. Substitute openai/gpt-4.1-mini to run the earlier comparator. Rendering uses OpenGL. On a headless Linux host, configure an appropriate MuJoCo GL backend (for example MUJOCO_GL=egl with a working EGL driver), or use --no-render in reproduce.py.

What the models control

Each decision cycle uses two sequential model calls:

1. Intent: approach, grasp, lift, carry, lower, release, withdraw or finish.
2. Motor channels: X/Y/Z each select negative, hold or positive; the gripper selects open, hold or close.

The model selects direction, not arbitrary absolute XYZ coordinates or joint torques. The shared executor scales step magnitude to 18, 4 or 2 mm near geometric targets, preserving the selected signs. It uses fixed-orientation inverse kinematics, workspace bounds and a 0.32-second physics increment.

Observations are structured simulator geometry, signed errors, alignment checks and contact feedback. They are not camera perception. Detailed action criteria are included in the prompts. No VLA, complete scripted pick-and-place skill, object teleportation or injected random grasp failure is used in these runs.

Physical success requires the apple to rest in the plate for at least 0.4 seconds, the gripper to be open, and TCP height to reach at least 145 mm. The prompt's withdraw/finish guidance is 160–170 mm; the physical stop threshold is the actual 145 mm criterion in the recorded implementation.

Model configuration and probabilities

• Jev: requested typesafe/jev-1.13, resolved to typesafe/jev-1.13-20260917 in the latest pair; native decisions endpoint.
• GPT-6: openai/gpt-6-astra, reasoning_effort=low, max_completion_tokens=4096, no temperature parameter, strict JSON schema.
• Mini: openai/gpt-4.1-mini, temperature=0, max_tokens=850, strict JSON schema.
• Requests use OpenRouter. Latest code uses a 90-second HTTP timeout. The earlier mini archive used 35 seconds; no published trial ended in an API error.

Jev bars are native model probability outputs. GPT bars are self-reported numbers in JSON. Neither is a calibrated probability of physical task success, and the values should not be compared as confidence scores across models.

Repository map

Path	Purpose
incremental_env.py, simulation.py	Scene, contacts, incremental physics, IK
incremental_policy.py	Full prompts, criteria, provider adapters, validation
incremental_run.py, reproduce.py	Online trials and response/trajectory logs
incremental_pair.py, incremental_compare_app.py	Concurrent paired live runs
incremental_triple_*	Read-only three-controller archive replay
verify_replay.py, test_incremental.py	Offline physical replay and tests
check_manifest.py, MANIFEST.sha256	Code, recording and asset integrity
incremental-*-comparisons/	Original records, videos, source snapshots
robot/	Vendored xArm7 model and meshes, with original BSD license
docs/, verification/, media/	Results, audit reports and rendered video

Original source snapshots are kept byte-for-byte for hash verification. The public runtime extracts only the IK helper and environment loader from earlier demo modules; physical replay verifies that this packaging preserves behavior.

License and attribution

OpenRoboto code: MIT. Robot description/meshes: UFACTORY BSD-3-Clause, via MuJoCo Menagerie; see third-party notices.

Inspired by the Jev Cartesian-control demonstration. This repository contains the simulation experiment and API adapters, not Jev model weights or real-robot deployment code.