jev-reranker: Relevance Filtering & Reranking for RAG

rerank(query, documents, *, instruction=None, threshold=0.0,
       top_k=None, return_documents=True, detail=False)
relevance_rerank(query, documents, *, instruction=None, threshold=0.2,
                 top_k=None, return_documents=True, detail=False)

query is a string and documents is a sequence of strings. The library accepts text documents, not image payloads. All ranking methods return a dictionary with a results list. The following is an illustrative shape, not a measured score:

{"results": [{"document_index": 1, "score": 0.9, "text": "Original document text"}]}

With detail=True, every method adds a top-level detail with the complete execution record, and each returned result also contains document-level details. If every candidate is filtered out, the response still contains "results": [] and the requested top-level details, including excluded documents.

Sort by descending score, preserving input order for ties. Duplicate texts remain separate candidates. Text in results is untruncated. Apply the threshold before top_k; top-k does not reduce scoring work. Empty input and top_k=0 make no requests. Unknown kwargs and invalid inputs raise errors rather than being ignored.

relevance_rerank() supports listwise and pointwise, not pairwise.

For full relevance diagnostics, request details:

from jev_reranker import JevReranker

reranker = JevReranker(api_key="YOUR-TYPESAFE-API-KEY...")
response = reranker.relevance_rerank(
    "Which planet is called the Red Planet?",
    ["Mars is called the Red Planet.", "Bread is made from flour."],
    threshold=0.2,
    detail=True,
)

response["detail"]["documents"] includes excluded scores and passes_threshold. response["detail"]["selection"] records counts before and after selection.

Install from PyPI:

uv add jev-reranker
# Or: pip install jev-reranker

Set TYPESAFE_API_KEY in .env or the environment. An explicit api_key= takes precedence over the environment, which takes precedence over .env. Keep .env out of version control. Set dotenv_path=None to disable file loading.

By default, lengths use Python's len(text); no tokenizer dependencies or downloads are required. Select extras as needed:

uv add 'jev-reranker[all]'
# Tokenizer only: uv add 'jev-reranker[tokenizer]'
# Sentence Transformers only: uv add 'jev-reranker[sentence-transformers]'

The all extra installs tokenizer dependencies, Sentence Transformers (including PyTorch), and evaluation-only pyarrow. The base installation does not include PyTorch. Model weights are downloaded when used, not when installing the extra.

Use api_key_env= to read a different environment variable name.

relevance_rerank() selects its preset by mode: the default listwise uses RELEVANCE_INSTRUCTION; explicit pointwise uses POINTWISE_RELEVANCE_INSTRUCTION. The latter judges each document without assuming access to other candidates. Both default to threshold 0.2, and an explicit per-call instruction= takes precedence. rerank() uses RERANK_INSTRUCTION for listwise and pointwise scoring. It asks whether a document helps answer the query and prefers specific facts, without prescribing numeric score anchors.

The relevance presets add explicit anchors: topic overlap alone should receive 0.1, and unrelated content or the wrong referent should receive 0.0. Useful partial answers and linking facts receive higher anchors. These instructions are designed to separate useful evidence from documents to discard, rather than preserve fine distinctions among weak candidates. As a result, they can work better for threshold filtering while being less suitable for ordering every candidate. Actual scores still depend on the model and context; the library does not force scores to these anchors or rescale them after the API response.

The instruction controls scoring; the threshold controls selection after scoring. Changing threshold alone does not change the prompt or the scores. For example, rerank(..., threshold=0.2) keeps the ordinary ranking prompt, while relevance_rerank(..., threshold=0.0) keeps the relevance prompt and returns all scored candidates. For the same supported mode, explicit instruction, and threshold, both methods use the same scoring and selection logic.

from jev_reranker import JevReranker

reranker = JevReranker(api_key="YOUR-TYPESAFE-API-KEY...", mode="pointwise")
results = reranker.relevance_rerank(
    "Which planet is called the Red Planet?",
    ["Mars is called the Red Planet.", "Bread is made from flour."],
)

The pointwise prompt judges each document using only that document and the query. Use detail=True to inspect the prompt and scores, and check the threshold when switching between listwise and pointwise.

RERANK_INSTRUCTION, PAIRWISE_INSTRUCTION, RELEVANCE_INSTRUCTION, and POINTWISE_RELEVANCE_INSTRUCTION in instructions.py are ordinary dictionaries. You can provide the same structure:

custom = {
    "instructions": "Does {document} contain concrete evidence for `query`?",
    "criteria": {
        "true": "Contains concrete supporting facts",
        "false": "Contains no supporting facts",
    },
}
reranker = JevReranker(api_key="YOUR-TYPESAFE-API-KEY...")
results = reranker.rerank("query", ["document"], instruction=custom, threshold=0.3)
# JevReranker(api_key="YOUR-TYPESAFE-API-KEY...", instruction=custom) sets the instance default.

Only the keys instructions and criteria are accepted. Criteria must contain nonempty true and false strings. Templates use {document}, or {left} and {right} for pairwise. Unknown keys and invalid placeholders fail before any request. Each call copies its instructions without mutating instance configuration, keeping concurrent calls separate. Legacy constructor arguments instructions= and criteria= remain supported but cannot be combined with constructor instruction=. relevance_rerank() selects the relevance preset independently of the constructor's ordinary ranking prompt; its per-call instruction= can override that preset.

HTTP uses httpx.AsyncClient and asyncio internally:

import asyncio
from jev_reranker import JevReranker

async def main():
    reranker = JevReranker(api_key="YOUR-TYPESAFE-API-KEY...", mode="pointwise", max_concurrency=8)
    results = await reranker.a_rerank("The Red Planet?", ["Mars.", "Venus."])
    response = await reranker.a_rerank(
        "The Red Planet?", ["Mars.", "Venus."], detail=True,
    )
    # Concurrent queries can share the same instance.
    batches = await asyncio.gather(
        reranker.a_rerank("The Red Planet?", ["Mars.", "Venus."]),
        reranker.a_rerank("Earth's satellite?", ["The Moon.", "The Sun."]),
    )
    print(results, response["detail"]["usage"], batches)

asyncio.run(main())

Sync and async methods share arguments, result structure, and scoring logic, including top_k, return_documents, and detail.

Ordinary instances can be reused across sync calls, caller threads, and async event loops. Each synchronous call uses a temporary asyncio.run() loop; no persistent loop thread is kept. Calling a sync method inside a running loop still raises ConfigurationError; use the async method instead.

Each call owns a separate, lazily created HTTP client. Success, failure, and cancellation all close it before the call finishes. A context-local call state keeps concurrent requests separate. No explicit shutdown is needed for normal use.

HTTP concurrency is bounded across the entire instance. Pointwise uses worker tasks, pairwise processes bounded groups of tasks, and listwise chunks run sequentially. Retries use asyncio.sleep. Synchronous tokenizer loading and counting use asyncio.to_thread.

Cancellation drains child tasks before propagating asyncio.CancelledError. HTTP failures also drain children and preserve public JevError exceptions rather than wrapping them in ExceptionGroup. aclose() and close() remain optional compatibility methods: they reject new work and wait for active calls, whose resources are cleaned up automatically. Cancel caller tasks first if you need to stop active work early.

document_max_length defaults to 4000. The default len(text) counts Unicode code points, not bytes or display width. Documents exceeding the limit are sent as prefixes; returned text remains unchanged.

reranker = JevReranker(api_key="YOUR-TYPESAFE-API-KEY...", document_max_length=8000)
results = reranker.rerank("query", ["document"])
# document_max_length=None disables document truncation.

For production use, consider token counting. English text in particular often has many more characters than tokens, so character limits can truncate documents or split requests earlier than needed. Install the optional tokenizer dependencies:

uv add 'jev-reranker[tokenizer]'

reranker = JevReranker(
    api_key="YOUR-TYPESAFE-API-KEY...",
    tokenizer="google/embeddinggemma-300m",
    document_max_length=8000,  # 8000 tokens here; the default is 4000 tokens.
    split_state_budget=26000,
    split_request_budget=48000,
)
results = reranker.rerank("query", ["document"])

Only the tokenizer is fetched on first scoring. Gemma license acceptance and Hugging Face authentication such as HF_TOKEN may be required. No model weights, PyTorch, or GPU are needed. The default Gemma revision is pinned; override it with split_tokenizer_revision="main", for example. Other Hub repositories, local tokenizer.json files, and objects implementing encode(text) / decode(ids) are supported. Supplying your own object does not require this library's tokenizer extra.

You can also provide a synchronous length_fn: Callable[[str], int], such as a UTF-8 byte counter:

def utf8_length(text: str) -> int:
    return len(text.encode("utf-8"))

reranker = JevReranker(api_key="YOUR-TYPESAFE-API-KEY...", length_fn=utf8_length, document_max_length=8000)
results = reranker.rerank("query", ["document"])  # Prefix of at most 8000 bytes.

The function must return a deterministic nonnegative integer and cannot be combined with tokenizer. It may run in a worker thread and receives both documents and serialized JSON state/questions/requests. Custom counters use binary search over text prefixes, followed by a bound check; the longest possible prefix is not guaranteed for nonmonotonic functions.

split_state_budget (26000) and split_request_budget (48000) use the same selected units. Queries are not truncated. Character counts, custom counts, and Gemma tokens are not guaranteed to match Jev's internal tokens; API context errors are handled separately. Legacy names document_max_tokens, split_state_token_budget, and split_request_token_budget remain aliases and do not force token-based measurement.

reranker = JevReranker(
    api_key="YOUR-TYPESAFE-API-KEY...",
    model="jev-latest",  # Override to select another Jev model.
    mode="listwise",
    document_max_length=4000,  # None disables prefix truncation.
    split_state_budget=26000,
    split_request_budget=48000,
)
results = reranker.rerank("query", ["document"])

Explicitly supplied clients and transports are caller-owned and are never closed by the reranker. A borrowed client intentionally overrides per-call connection ownership, remains bound to its first event loop, and requires async methods. Custom transports must support the caller's concurrency and loop usage. These advanced options are unnecessary for ordinary API calls.

Gemma length settings estimate Jev input size; they do not extend the EmbeddingGemma model's context. Queries and questions also consume the budgets. Partitioned listwise scoring uses different shared contexts and is not guaranteed to produce the same scores as an unsplit request. Pairwise uses n(n−1)/2 requests and is most suitable for small candidate sets.

Named tokenizers disable truncation and padding and count the full input even beyond 65536 tokens. Supplied objects with writable model_max_length are adjusted to at least tokenizer_max_length, mutating that object. Objects without this attribute are left unchanged. This does not change document_max_length.

Custom encoders should not add special tokens or truncate input. Wrap Transformers tokenizers to pass add_special_tokens=False, truncation=False. Custom question templates must contain {document} for listwise/pointwise or {left} and {right} for pairwise. Other fields and unknown kwargs are rejected.

reranker = JevReranker(
    api_key="YOUR-TYPESAFE-API-KEY...",
    instructions="Does {document} directly answer `query`?",
    criteria={"true": "Provides the requested facts", "false": "Does not provide the requested facts"},
)

import json
from pathlib import Path
from jev_reranker import JevError, JevReranker

reranker = JevReranker(api_key="YOUR-TYPESAFE-API-KEY...")
try:
    response = reranker.rerank("The Red Planet?", ["Mars.", "Venus."], detail=True)
except JevError as exc:
    # Inspect APIError.status_code or partial execution details in exc.detail.
    raise

Path("rerank-log.json").write_text(
    json.dumps(response, ensure_ascii=False, indent=2), encoding="utf-8",
)

response["detail"] records effective settings, Python/dependency versions, requested/resolved models, tokenizer revision, usage, retries, splits, submitted state/questions, and responses. Per-result detail contains original_length, sent_length, length_unit, related request IDs, and pairwise comparisons. rerank(..., detail=True) and relevance_rerank(..., detail=True) return the same full execution record, including excluded documents.

Detail schema_version is 2. usage.input_tokens and output_tokens are API-reported token counts, distinct from local length estimates. Logs include document text but exclude API keys, authentication headers, and arbitrary environment variables.

HTTP 429/500/502/503/504/529 and transport errors receive exponential backoff with jitter, respecting Retry-After up to 60 seconds. Authentication failures and malformed successful responses are not retried. Listwise context errors repartition only the failed candidates. A document or pair that cannot fit raises ContextLimitError. Failures are never converted to zero scores. Other public exceptions are ConfigurationError, APIError, and ResponseValidationError, all derived from JevError.

Do not mutate constructor settings during use; create another instance for different settings.