Contributing

August 25, 2026 · View on GitHub

Thanks for considering a contribution. RewardHarness is a research codebase, so we keep the contribution rules short — but a few are non-negotiable.

Before you open a PR

  1. Open an issue first for non-trivial changes (new features, refactors, anything > 50 lines). Use the bug report or feature request template. This avoids two-person work and surfaces design feedback before the code is written.
  2. Run the preflight + tests locally: 
    make install-dev
make check    # env / credentials / endpoints
make test     # mocked tests, ~2 s
make quality  # Ruff, mypy, rating-integrity and shell syntax gates
    PRs that break the test suite won't be merged.
  3. Keep PRs focused. One PR = one logical change. If you're refactoring while adding a feature, split into two PRs.

Commit messages

Conventional-commit-ish, lowercase type prefix:

feat:  ...   new user-visible behavior
fix:   ...   bug fix
docs:  ...   docs / README / comments
chore: ...   tooling, deps, formatting
test:  ...   tests only
ci:    ...   CI config only
deps:  ...   dependency changes
refactor: ... no behavior change

One-line subject (≤72 chars), then optional body. Reference issues with #NNN.

What we do NOT add to commits

  • No Co-Authored-By trailers. This is a strict project rule. If you authored a commit, sign it with your own name only; if you authored it as a team, agree offline on a single author. The internal dev repo enforces this and we keep the public repo consistent.
  • No "Generated by …" / AI tool attribution footers. Same reasoning: the author field is the source of truth.
  • No reflowed unrelated whitespace (don't include git diff -w-clean noise in a feature PR).

Code style

  • Python 3.10–3.13. Type hints are required for runtime code under rewardharness/ and encouraged for one-off scripts.
  • Don't introduce new dependencies unless you're prepared to argue they're worth the install cost in the PR description.
  • Default to no comments. Add one only when the why is non-obvious (a hidden constraint, a workaround, a surprising behavior).

Tests

  • All new code in rewardharness/ should have a corresponding mocked test in tests/.
  • Tests must not make real network calls. Inject or patch the client boundary; see tests/README.md for the current patterns.

Releases

Versions follow SemVer. Before tagging, maintainers run make release-check, cut a tag + GitHub Release, and add a ## [x.y.z] block to CHANGELOG.md.

Release candidates use PEP 440 versions such as 0.2.0rc1 and matching tags such as v0.2.0-rc1. The tag workflow publishes the same validated artifacts to PyPI and GitHub through trusted publishing. Run rewardharness release-status and python scripts/check_release_metadata.py before tagging. make release-check also runs scripts/check_distribution.py to reject stale artifacts, metadata drift, and missing package resources; the workflow then installs the published PyPI artifact and verifies the resulting GitHub Release before reporting success.