stats/

Node.js does the data wrangling (normalization, distributions, divergences); R does the inference and figures (clustering, classification, ROC/EER). Steps can run one at a time or all at once; every step writes its intermediate result to disk, so downstream steps never recompute upstream work.

Pipeline

data/runs/*/responses.jsonl
  → 01-normalize.js        → data/derived/normalized.jsonl (+ normalization-report.json)
  → 02-distributions.js    → results/distributions.json
  → 03-divergence.js       → results/divergence.json, divergence-matrix.csv, split-scores.json
  → R/10-clustering.R      → results/clustering.json, figures/dendrogram.pdf
  → R/11-classification.R  → results/classification.json
  → R/12-verification-roc.R→ results/verification.json, figures/roc.pdf, budget-curve.pdf
  → 04-pilot-report.js     → results/pilot-report.md   (pilot go/no-go, reads all of the above)

Run everything: npm run stats:all · Single steps: npm run stats:normalize, stats:distributions, stats:divergence, stats:clustering, stats:classification, stats:verification, stats:pilot-report. R deps (jsonlite, pROC): npm run stats:r-deps.

Method mapping to paper1 (keep in sync with paper1/main.tex §Method)

Analysis	Script	Paper concept
Answer normalization & validity taxonomy	01	§IV-B data processing
Empirical distributions, entropy, mode share	`02$	\text{fingerprint} \text{definition}
\text{Pairwise} \text{Jensen}-\text{Shannon} \text{divergence} (\text{mean} \text{over} \text{task} \times \text{lang} \text{cells}, \text{min} \text{n}=10)	$03`	fingerprint distance
Split-half genuine/impostor trials (rep parity split)	03	verification trial construction
UPGMA clustering, cophenetic corr., ARI vs. family labels	R/10	RQ2 lineage recovery
Leave-one-out 1-NN family classification	R/11	RQ2 accuracy; misclassifications = anomaly candidates
ROC/AUC/EER + query-budget curve (fixed seed 20260704)	R/12	RQ3 verification protocol
Pilot go/no-go (validity, determinism, family separation permutation test, cost extrapolation)	04	§pilot

Conventions

• Nothing is silently dropped. Refusals/off-format answers are coded (answer_class) in step 01 and excluded explicitly downstream (valid filter); the normalization report counts every exclusion.
• Determinism. All stochastic steps (permutation test in 04, subset resampling in R/12) use fixed seeds recorded in outputs.
• Colors. Cross-language color analysis needs color-lexicon.json; step 01 reports unmapped answers — extend the lexicon and re-run (this is data curation, document it).
• Adding an analysis: new numbered script, one output file in results/, entry in run-all.sh + package.json + the table above.

Paper2 analyses — TODO (do not build before paper1 is submitted)

Ideas parked for the coordination study (see paper2/README.md for the full framing):

• Coordination rate per task×lang: expected matching probability from independent samples, Σp² (pseudo-pairs — no physically paired queries needed), against the cross-model marginal baseline Σ(p·q̄).
• Intra- vs inter-family coordination gap (H2), mixed-effects model with task and language as random effects.
• H3: correlation between fingerprint JSD (paper1 metric) and pairwise coordination rate — Mantel test on the two matrices.
• Anticoordination: deviation from own prior (KL between coordination and anticoordination distributions) as a measure of counterpart modeling.
• Secrecy condition: refusal-rate analysis + shift of password distributions vs. coordination distributions; entropy as "password guessability" (security framing).
• Cross-language color convergence: JSD between a model's color distributions across languages, via color_canon.