JSON Interchange Format

Create and consume language-neutral calibration datasets without knowing Swift's enum representation.

Overview

The judge-calibration/v1 schema is intended for annotation tools, Python pipelines, databases, Swift applications, and the judge-calibrate CLI. Stable sample, rater, and dimension identifiers are JSON strings. Rating values are ordinary JSON Booleans, finite numbers, or strings.

A scored ordinal rating looks like this:

{
  "sample": "book-01",
  "rater": "expert",
  "dimension": "relevance",
  "outcome": { "status": "scored", "value": 4 },
  "metadata": {}
}

An explicit abstention is different from a missing rating:

{
  "sample": "book-07",
  "rater": "judge-v1",
  "dimension": "relevance",
  "outcome": {
    "status": "abstained",
    "reason": "The result description is incomplete."
  },
  "metadata": {}
}

Declare every dimension's measurement scale. For example:

{
  "id": "relevance",
  "name": "Book-search relevance",
  "scale": { "type": "ordinal", "values": [1, 2, 3, 4, 5] },
  "metadata": {}
}

The complete hand-authored files in the repository's Examples directory are executable documentation. The schema never contains compiler-generated keys such as _0 or Swift wrapper storage such as rawValue.

Compatibility rule

Adding an optional field is compatible. Changing a field's meaning, removing a field, or changing one of the documented shapes requires a new schema version. Validation rejects unsupported top-level schema versions rather than guessing.