Contributing

Thanks for helping make judge calibration in Swift more trustworthy.

Development

The JudgeCalibration core target links no third-party modules. The root package resolves xceval's graph-wide XCEvalFormat dependency:

swift test

The sibling CLI intentionally has its own dependency graph:

cd CLI
swift test

Run full builds at integration boundaries rather than after every small edit. Add focused tests for changed behavior first.

Before submitting, run the format and documentation gates used by CI:

swiftformat --lint .
bash Scripts/build-documentation.sh

The documentation script uses the docc compiler bundled with Xcode and emits the required symbol graph itself; the root package does not depend on the DocC SwiftPM plugin.

Statistical changes

Every metric change must include:

• its exact supported measurement scales;
• its undefined and degenerate conditions;
• a canonical worked example or a fixture from a pinned independent oracle;
• tests for missing values, constants, ties, and non-finite input where relevant;
• evidence that unavailable cases never become zero or NaN.

Bootstrap changes must continue to resample whole sample clusters and remain deterministic for a public seed.

Evaluations framework changes

Never hardcode an Xcode beta path. Run:

bash Scripts/check-evaluations-interface.sh
bash Scripts/build-evaluations-adapter.sh

If the interface check finds a new agreement/calibration API, stop and review the package boundary before adapting to it. Record changed evidence in docs/DESIGN.md.

The model-free runtime adapter test in Tests/JudgeCalibrationEvaluationsTests runs through the package's own JudgeCalibrationKit-Package scheme, driven by Scripts/check-evaluations-simulator.sh. It deliberately has no companion .xcodeproj and no application host: a SwiftPM test bundle runs hostless, and on iOS 27 an app built against the SDK must adopt the UIScene life cycle or it will refuse to launch, taking the test bundle with it. Do not reintroduce a host app.

Interchange compatibility

Do not parse Apple's private persisted JSON representation. Changes to the normalized xceval importer must name a tagged xceval release and preserve unknown-field tolerance within a supported schema version.

The public judge-calibration/v1 and judge-calibration-report/v1 formats must not expose Swift synthesized keys such as _0 or rawValue. Update the hand-authored interchange fixture and examples for compatible additions. Use a new schema version for incompatible changes.

Pull requests

Keep public APIs small, document new public symbols, and explain statistical or compatibility tradeoffs in the pull request. Update the changelog for user- visible changes.