Contributing

May 15, 2026 · View on GitHub

Thanks for considering a contribution. RewardHarness is a research codebase, so we keep the contribution rules short — but a few are non-negotiable.

Before you open a PR

  1. Open an issue first for non-trivial changes (new features, refactors, anything > 50 lines). Use the bug report or feature request template. This avoids two-person work and surfaces design feedback before the code is written.
  2. Run the preflight + tests locally: 
    make check    # env / credentials / endpoints
make test     # 100 mocked tests, ~2 s
    PRs that break the test suite won't be merged.
  3. Keep PRs focused. One PR = one logical change. If you're refactoring while adding a feature, split into two PRs.

Commit messages

Conventional-commit-ish, lowercase type prefix:

feat:  ...   new user-visible behavior
fix:   ...   bug fix
docs:  ...   docs / README / comments
chore: ...   tooling, deps, formatting
test:  ...   tests only
ci:    ...   CI config only
deps:  ...   dependency changes
refactor: ... no behavior change

One-line subject (≤72 chars), then optional body. Reference issues with #NNN.

What we do NOT add to commits

  • No Co-Authored-By trailers. This is a strict project rule. If you authored a commit, sign it with your own name only; if you authored it as a team, agree offline on a single author. The internal dev repo enforces this and we keep the public repo consistent.
  • No "Generated by …" / AI tool attribution footers. Same reasoning: the author field is the source of truth.
  • No reflowed unrelated whitespace (don't include git diff -w-clean noise in a feature PR).

Code style

  • Python 3.10+. Type hints encouraged but not required for one-off scripts.
  • Don't introduce new dependencies unless you're prepared to argue they're worth the install cost in the PR description.
  • Default to no comments. Add one only when the why is non-obvious (a hidden constraint, a workaround, a surprising behavior).

Tests

  • All new code in src/ should have a corresponding mocked test in tests/.
  • Tests must not make real network calls. If you need a Gemini response, patch src.router.call_gemini. If you need a Sub-Agent completion, patch src.sub_agent.OpenAI. See tests/README.md for the patterns.

Releases

Versions follow SemVer. Maintainers cut a tag + GitHub Release and add a ## [x.y.z] block to CHANGELOG.md.