Foreman

September 17, 2026 · View on GitHub

Jev supervises a Codex coding worker through Vercel AI Gateway, while a programmer-selected acceptance command checks the result. This is an experimental derivative of thruwire/foreman, with the upstream MIT license retained.

Start here: programmer setup and first task.

This repository is public. Anyone can clone and try it; no organization invitation is needed.

The 0.1.1+shifty.3 public release preserves the runtime from the 0.1.1+shifty.2 pilot, which passed a fresh macOS install and a real Codex/Jev coding run. See the tested release and team handoff for evidence and platform limits.

For Azure DevOps reviews, use the read-only PR review workflow. It collects an immutable diff, accepts a programmer/ChatGPT review draft, and asks Jev to assess its evidence. It creates local reports and does not launch a reviewer, approve, or merge a PR.

Quick start

Requirements: Git, Python 3.11+, macOS/Linux (Windows users: WSL2). Real runs also need an authenticated Codex CLI and a provisioned Vercel AI Gateway key with credits and Jev access. The examples use Python 3.12; substitute another installed Python 3.11+ interpreter if needed. Check its version first: macOS's default python3 may still be Python 3.9.

git clone https://github.com/Shifty-Eye-Games/foreman-jev.git
cd foreman-jev
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install '.[dev]'
python -m pytest -q
foreman demo --repo .

Tests and the demo run offline. For a real pilot, copy .env.example to .env, configure your own key or external key-file path, then run:

foreman check-provider
foreman run \
  --repo /path/to/disposable-git-repository \
  --job 'Fix the small reproducible bug and preserve the acceptance tests.' \
  --verify 'python -m pytest -q'

check-provider uses a paid synthetic Jev request. run uses paid Jev assessments and the programmer's Codex access. The guide includes a dependency-free practice repository, Windows/WSL notes, configuration, budgets and an evaluation checklist.

Changes from upstream

  • Vercel AI Gateway adapter for typesafe-ai/jev, with validated probabilities, bounded retries, deadlines and sanitized API errors. The original TypeSafe provider remains available.
  • Mandatory acceptance command: optimistic model scores and a successful Codex exit cannot bypass verification. The runtime executes the operator's command and records its exit code.
  • Source fingerprints invalidate checks if source changes during verification or before finish.
  • Evidence covers staged, unstaged, newly created and worker-committed changes since the run began.
  • Common credential paths and recognizable secrets are filtered from observations and saved state.
  • Sanitized observation snapshots are recorded for auditing; Git worktrees correctly ignore logs.
  • Offline regression tests and CI on Linux and macOS.
  • Azure PR snapshots with bounded evidence packets, line-selected supporting context, live revision checks, and separate audit directories for successful and failed assessment attempts.

What completion means

FINISHED requires a zero exit code from the configured acceptance command on unchanged observed source, plus Jev's completion thresholds. It does not certify the test suite, correctness or readiness to merge. A worker can weaken tests before checking them; review all changes and keep normal project CI and human review.

The pilot is for small code repositories. Native Windows process control, CC5/browser automation, large Unity asset trees, hermetic builds and reliable dollar-cost accounting are outside this release. No speed or quality improvement over ordinary Codex has been established.

Commands

foreman --help
foreman check-provider
foreman demo --repo .
foreman runs --repo /path/to/project
foreman inspect RUN_ID --repo /path/to/project
foreman snapshot-azure --help
foreman assess-pr --help

Run data is stored in the target repository's ignored .foreman/runs/ directory. Ctrl-C stops the run. inspect reads a timeline; there is no resume command.

Development

python -m pytest -q
ruff check .
python -m pip install build
python -m build

Reinstall with python -m pip install '.[dev]' after changing installed CLI code. Pytest reads src/ directly. CI also smoke-tests a built wheel outside the checkout. Tests use synthetic keys, fake models and local subprocesses; no real model credentials are needed.