Contributing

Useful contributions include reproducible failures, clearer examples, compatible model backends, and evaluations on independently labeled real tasks.

Report Python/package versions, model revision, question, label descriptions, minimal input, and expected versus actual behavior. Remove private data. For model errors, explain why the expected label fits the rubric; a surprising prediction is not necessarily a library bug.

uv venv --python 3.12
uv pip install --python .venv/bin/python -e '.[dev]'
.venv/bin/python -m pytest -q

Tests need no MLX, model weights, or credentials. API tests use httpx.MockTransport; they do not establish live provider compatibility or model quality. Install the mlx extra on Apple Silicon for inference. Keep calibration/test data separate; publish failures with successes and document model, hardware, prompts and repetitions for timing.

Code is MIT licensed. Include third-party code/data only when redistribution is permitted and required attribution is supplied.