scored-rerank

June 10, 2026 · View on GitHub

Outcome-scored reranking for Rust — rank candidate results by what reality confirmed, not just by base similarity.

crate license unsafe

A retriever or recommender assigns every candidate a base relevance score (BM25, cosine similarity, a cross-encoder logit, …). That score says how well an item matches a query a priori — but it says nothing about whether the item actually earned its keep: did the user click it, did it resolve the ticket, did the answer turn out correct?

scored-rerank closes that loop. Each item carries a Beta-Bernoulli posterior over its hidden good-outcome rate, updated from binary feedback. At rank time the base score is blended with the learned posterior so that items which repeatedly earn good outcomes rise, items that look good but disappoint sink, and items with little evidence stay near their base score.

  • Zero dependencies by default — deterministic posterior-mean reranking.
  • #![forbid(unsafe_code)] — entirely safe Rust.
  • Optional Thompson-sampling exploration behind the thompson feature (pulls in rand).

The method

Beta-Bernoulli posteriors

Model each item's "was this a good outcome?" indicator as a Bernoulli(θ) with unknown success probability θ. The conjugate prior of the Bernoulli is the Beta(α, β) distribution, so updating from feedback is just incrementing a count:

good outcome (reward = true)   →  α += 1
bad  outcome (reward = false)  →  β += 1

Two quantities fall out of the posterior:

quantityformulameaning
meanπ = α / (α + β)estimated good-outcome rate
confidenceα + βtotal evidence (pseudo-count)

A uniform prior Beta(1, 1) starts every item at π = 0.5 — neutral until evidence arrives.

The blend

rank combines the (normalized, [0, 1]) base score with the posterior mean as a convex combination controlled by base_weight ∈ [0, 1]:

blended = base_weight · base_score + (1 − base_weight) · posterior_mean

We deliberately use the convex blend rather than a multiplicative base · mean: the product collapses to zero whenever either factor is zero, so a single early failure (or a base score of 0) would permanently bury an item. The convex blend keeps both signals bounded in [0, 1] and lets you tune how much confirmed outcomes are allowed to move the ranking — base_weight = 1.0 reproduces the original ranking, 0.0 ranks purely by what reality confirmed.

Thompson sampling (exploration)

Ranking by the posterior mean is greedy: an item that got unlucky early can be starved of the impressions it needs to recover. With the thompson feature, [Reranker::rank_thompson] draws one sample from each item's posterior instead of using the mean — classic Thompson sampling (Thompson, 1933). Items with little evidence have wide posteriors and occasionally sample high, earning exploratory exposure; well-evidenced items have tight posteriors and behave near-greedily. This is the standard, asymptotically near-optimal way to balance exploration and exploitation in a Bernoulli bandit (Russo et al., 2018).

Usage

[dependencies]
scored-rerank = "0.1"
# Optional exploration:
# scored-rerank = { version = "0.1", features = ["thompson"] }
use scored_rerank::Reranker;

// base_weight = 0.5 → weigh base similarity and confirmed outcomes equally.
let mut rr = Reranker::new(0.5);

let candidates = [("doc-a", 0.8_f64), ("doc-b", 0.8_f64)];

// Reality speaks: doc-b keeps earning good outcomes, doc-a keeps failing.
for _ in 0..10 {
    rr.record("doc-b", true);
    rr.record("doc-a", false);
}

let ranked = rr.rank(&candidates);
assert_eq!(ranked[0].0, "doc-b"); // confirmed winner rises above the tie

With exploration (--features thompson):

use rand::{rngs::StdRng, SeedableRng};

let mut rng = StdRng::seed_from_u64(42);
let ranked = rr.rank_thompson(&candidates, &mut rng);

See examples/rerank.rs for a runnable demo:

cargo run --example rerank
cargo run --example rerank --features thompson

API at a glance

  • BetaPosterior { alpha, beta }mean(), confidence(), variance(), update(reward: bool), and (with thompson) sample(rng).
  • Reranker::new(base_weight) / Reranker::with_prior(base_weight, prior).
  • record(item, reward: bool) — fold one outcome into the item's posterior.
  • rank(candidates) -> Vec<(ItemId, blended_score)> — deterministic.
  • rank_thompson(candidates, rng) — exploratory (feature thompson).
  • posterior(item), effective_posterior(item), tracked_items().

References

  • W. R. Thompson (1933). On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25(3/4), 285–294.
  • D. Russo, B. Van Roy, A. Kazerouni, I. Osband, Z. Wen (2018). A Tutorial on Thompson Sampling. Foundations and Trends in Machine Learning 11(1), 1–96. arXiv:1707.02038.

License

MIT — see LICENSE.