LangSearch Lab

A local, no-dependency playground for designing staged search pipelines — search, score with Jev, dive deeper into sources, score again — and for measuring whether a staging strategy actually beats a plain search. Built to develop ideas for @jamubc/opencode-langsearch against the real primitives that plugin uses.

Run it

node server.js        # open http://localhost:8787

No npm install. Node ≥ 18.

First launch opens a recorded specimen run — a complete search → dedupe → gate → fetch → trim pipeline on the question "What is the most dangerous place to be on earth during a nuclear war?", assembled with the same engine functions the app executes live, so every record (parameters, full Jev request, raw answers, decisions, passage splits, char counts) is internally consistent. It is labeled as a specimen, costs nothing, exercises every view, and can be branched, re-run live, compared against your own runs, or deleted. The protocol for building your own run is printed on the empty Pipeline screen; both keys can be verified in place with the test key buttons.

The idea it exists to test

A plain LangSearch call returns whatever the engine ranks. A staged run instead does something like:

search → dedupe → Jev gate (rank + filter) → fetch the top pages in full
       → Jev trim (keep only query-bearing passages) → search again, refined

Each stage transforms a working set; every execution is recorded with its request, its metrics, and the set it produced. Branch any run at any stage to try a different continuation from the same point, then compare runs on the same question: tokens, cost, wall time, final chars, and URL-level overlap between the final sets.

Stages

Stage	What it does	Cost
search	LangSearch POST /v1/web-search — full request contract (count 1–50, freshness presets/dates/ranges, include/exclude domains, snippet / text:true / maxCharacters). replace or collect into the working set	free; daily allowance resets 00:00 UTC
dedupe	Local duplicate collapse — URL canonicalisation + 5-word shingle containment ≥ 0.8 (print variants, mirrors, shared boilerplate)	free, no network
gate	Jev scoring of every result. filter drops failing results, ranks the rest (relevant → evidence), caps, applies the recency cutoff and fallback rules; observe only records scores — the calibration mode. Checks, bounds and question text are all editable; blank bound = observe that check	$0.042 / 1M input tokens
jev	Custom questions playground — all three System One primitives. noul (is it true? → probability), score (which level? → position 0..levels−1 with confidence), choice (which of these? → option key with confidence). Compose the payload yourself: results or passages as the state, per-result or once-per-set questions, presets (partisanship, source-type, contribution, …) or your own. Per-result answers attach under their own id — numerics in score columns, choice labels in a label column — and flow into the working set, Compare and both exports. Optionally re-sorts the working set by a noul/score question	$0.042 / 1M input tokens
fetch	Reads selected result URLs server-side, extracts page text (up to 100 000 chars — past LangSearch's 5000 cap) and replaces each result's content. Original search text is kept. This is the dive-deeper step	free
trim	One Jev question per passage of every result; keeps passages above threshold (each result keeps at least its best). dry run scores without changing text	$0.042 / 1M input tokens

Fidelity to the plugin

The Jev request shape, the check wording (relevant, evidence, injection, slop, agreement, timely), the trim question, the decision rules (precedence, over-cap, injection-safe fallback, stale cutoff, undated-never-dropped) and every default (count 8, maxContentChars 1500, maxResults 4, relevant ≥ 0.45, evidence ≥ 0.5, injection ≤ 0.5, minTimely 0.5, maxAgeDays 180, trim 0.5, dedupe 0.8) are ported from opencode-langsearch's src/index.ts and src/checks.ts, so an experiment here transfers 1:1 to the plugin. Check ids may not contain underscores; passage is reserved.

Missing Jev answers read as 0 in gate decisions and 1 (kept) in trim decisions, exactly like the plugin.

Writing custom Jev questions

The jev stage is the playground for everything the real one does — the three primitives from docs.typesafe.ai/primitives, in one request if you like:

Primitive	Question	criteria	Answer
noul	Is it true?	optional { true, false } clarification	noul — probability 0–1
score	Which level?	ordered array of levels (2–10)	score — position 0..levels−1 (can land between levels), plus legend, probabilities, confidence
choice	Which of these?	map { optionKey: description } (2–255)	choice — the selected key, plus probabilities, confidence

A per-result question is asked once per result with <key> replaced (results.r0, or passages.r0p3 in passages mode — whichever container you name is normalised to the one being sent, so one question works in both modes). A set question is asked once for the whole payload. Question ids may not contain underscores (they group the answers) and passage is reserved. Normalise a score by dividing it by its top level (levels.length - 1) before combining scales.

Two things the plugin learned that the presets respect: Jev has no clock — never ask how old something is, compute age locally from datePublished instead; and questions must read literally, so the criteria, not the instruction, are what disambiguate answers.

Comparison

Tick cmp on two or more runs → Compare tab:

• aggregates with deltas vs baseline: LS tokens in/out, Jev tokens, cost, wall time, final results, final chars
• final-set overlap vs baseline: shared, unique, Jaccard
• URL matrix: each URL's rank in every run's final set

Keys and privacy

Both keys are pasted into the left panel and kept in the browser's localStorage only. Calls go through the local proxy (/api/search, /api/systemone), which forwards them as Authorization: Bearer … and never logs keys or bodies. The fetch stage refuses private/local hosts. Nothing leaves the machine except the API calls themselves.

Files

server.js          static server + three proxies (search / systemone / fetch)
public/index.html  workbench layout
public/styles.css
public/engine.js   ported primitives: dedupe, checks, request builders, decision logic
public/stages.js   stage forms, validation, executors
public/app.js      state, rendering, branching, comparison

Reference

The Reference tab in the app holds the LangSearch request contract, the System One request/response shape, the ported defaults and the cost table.