Machine-Readable Specs

Potato publishes two generated, machine-checkable contracts alongside the prose documentation. Both are produced from the running code rather than maintained by hand, so they describe what Potato actually does today.

If you are pointing a coding agent (Claude Code, Codex, Cursor) at Potato, these two files plus llms.txt are the highest-value things to give it.

Artifact	What it describes	URL
Config JSON Schema	Every valid key, annotation type, and display type in a task config.yaml	schemas/potato-config.schema.json
OpenAPI 3.1 spec	Every HTTP endpoint the server registers	api-reference/openapi.json
llms.txt	Curated index of the documentation	llms.txt
llms-full.txt	The entire documentation set in one file	llms-full.txt

---

Config JSON Schema

Validates a task config.yaml before you ever start the server. It is generated from the same registries the server validates against, so a newly registered annotation type appears in the schema immediately.

It currently covers 154 top-level config keys, 56 annotation types, and 23 display types.

Editor validation

Every config in examples/ carries a modeline that switches on live validation in VS Code (with the YAML extension), JetBrains IDEs, Zed, and Helix:

# yaml-language-server: $schema=https://potatoannotator.readthedocs.io/en/latest/schemas/potato-config.schema.json
annotation_task_name: My Task
task_dir: .

Add that line to your own configs to get autocomplete and inline errors.

Validating in CI

pip install check-jsonschema
check-jsonschema --schemafile \
  https://potatoannotator.readthedocs.io/en/latest/schemas/potato-config.schema.json \
  path/to/config.yaml

The schema also ships inside the installed package, so it resolves offline:

import json, importlib.resources as res
schema = json.loads(
    res.files("potato").joinpath("schemas/potato-config.schema.json").read_text()
)

What it catches

Typo'd or hallucinated annotation types (radioo, sentiment), missing per-type required fields (a constant_sum scheme without labels), unknown display types, invalid assignment_strategy values, a missing item_properties.text_key, and configs with no data source at all.

!!! note "Deliberately permissive about extra keys" additionalProperties is true throughout. Potato only warns about unrecognized config keys, so a schema that rejected them would be stricter than the server and would flag working configs as invalid.

---

OpenAPI 3.1 spec

An index of Potato's HTTP surface: 390 paths / 427 operations, generated by walking the live Flask url_map.

Two extensions carry information a plain OpenAPI document cannot:

• x-potato-auth — the decorators guarding the operation (admin_required, api_login_required, same_origin_required, login_required). Operations without it are unauthenticated.
• x-potato-requires-config — set when the endpoint only exists if a config flag is on. /admin/arena, for example, is registered only under arena.enabled: true. About a quarter of all operations are config-gated.

Example

# Every admin-only endpoint
jq -r '.paths | to_entries[] | .key as $p | .value | to_entries[]
       | select(.value["x-potato-auth"] // [] | index("admin_required"))
       | "\(.key|ascii_upcase) \($p)"' openapi.json

# Everything that needs datasets.enabled
jq -r '.paths | to_entries[] | .key as $p | .value | to_entries[]
       | select(.value["x-potato-requires-config"] // "" | test("datasets"))
       | "\(.key|ascii_upcase) \($p)"' openapi.json

Scope

This spec is an authoritative index of what exists, not a payload contract — request and response bodies are not modelled. For narrative descriptions of the main endpoints and their payloads, see the API Reference.

Most endpoints are session-authenticated: POST /auth with username and password as form data, then reuse the session cookie.

---

Regenerating

All four artifacts are generated, and CI fails if a checked-in copy drifts from the code:

python scripts/generate_config_schema.py     # config JSON Schema
python scripts/generate_openapi.py           # OpenAPI spec
python scripts/generate_llms_full.py         # llms-full.txt

# Verify without writing (what CI runs)
python scripts/generate_config_schema.py --check
python scripts/generate_openapi.py --check
python scripts/generate_llms_full.py --check

Run them after adding an annotation type, a display type, a config key, or a route. See Testing for the drift tests that enforce this.