Jev is odd: live benchmark
September 18, 2026 · View on GitHub
Measured 2026-09-18T12:32:40.152Z → 2026-09-18T12:32:47.089Z. Requested model: jev-1.13.0. Reported models: jev-1.13.0.
20/20 planned cases correct. 20 responses, 0 request failures, 0 not attempted. This is a small fixed sample, not a general arithmetic accuracy claim.
| Measurement | Observed result |
|---|---|
| Mean round-trip | 345.3 ms |
| p50 / p95 (nearest rank) | 294.3 / 364.8 ms |
| Input / output tokens | 6928 / 640 |
| Estimated successful-call cost | $0.00029098 |
Sequential client round-trip; includes network and parsing; no retries, caching, or excluded warmups. Correctness oracle: BigInt remainder.
Latency statistics include all successful responses, including wrong answers, and exclude failed requests. Each request's timestamp, outcome, and duration are retained below. Token totals cover successful responses only; failed requests have unknown usage. Estimate from reported successful input tokens and published list price; not an invoice. Failed requests may incur additional charges. Price checked 2026-09-18: $0.042/million input tokens; output tokens free. An unknown model has no price estimate. Runtime: v26.3.0, darwin/arm64.
| Integer | Jev says | Correct | Round-trip ms | Input tokens | Output tokens | Confidence | Started at (UTC) |
|---|---|---|---|---|---|---|---|
| 0 | even | yes | 1278.8 | 340 | 32 | 1.0000 | 2026-09-18T12:32:40.155Z |
| 1 | odd | yes | 270.0 | 340 | 32 | 1.0000 | 2026-09-18T12:32:41.436Z |
| 2 | even | yes | 285.7 | 340 | 32 | 1.0000 | 2026-09-18T12:32:41.707Z |
| 7 | odd | yes | 364.8 | 340 | 32 | 1.0000 | 2026-09-18T12:32:41.995Z |
| 42 | even | yes | 294.3 | 341 | 32 | 1.0000 | 2026-09-18T12:32:42.361Z |
| -1 | odd | yes | 311.3 | 341 | 32 | 1.0000 | 2026-09-18T12:32:42.656Z |
| -2 | even | yes | 296.4 | 341 | 32 | 1.0000 | 2026-09-18T12:32:42.968Z |
| -7 | odd | yes | 361.7 | 341 | 32 | 1.0000 | 2026-09-18T12:32:43.267Z |
| 100 | even | yes | 273.2 | 342 | 32 | 1.0000 | 2026-09-18T12:32:43.630Z |
| 101 | odd | yes | 268.9 | 342 | 32 | 1.0000 | 2026-09-18T12:32:43.905Z |
| 1000 | even | yes | 296.0 | 343 | 32 | 1.0000 | 2026-09-18T12:32:44.175Z |
| 1001 | odd | yes | 323.5 | 343 | 32 | 1.0000 | 2026-09-18T12:32:44.472Z |
| 2147483646 | even | yes | 281.2 | 349 | 32 | 1.0000 | 2026-09-18T12:32:44.797Z |
| 2147483647 | odd | yes | 264.7 | 349 | 32 | 1.0000 | 2026-09-18T12:32:45.080Z |
| -2147483647 | odd | yes | 266.2 | 350 | 32 | 1.0000 | 2026-09-18T12:32:45.345Z |
| 9007199254740990 | even | yes | 272.1 | 355 | 32 | 0.9600 | 2026-09-18T12:32:45.613Z |
| 9007199254740991 | odd | yes | 318.1 | 355 | 32 | 1.0000 | 2026-09-18T12:32:45.887Z |
| -9007199254740991 | odd | yes | 315.2 | 356 | 32 | 0.9900 | 2026-09-18T12:32:46.207Z |
| 900719925474099312345 | odd | yes | 257.1 | 360 | 32 | 0.7100 | 2026-09-18T12:32:46.523Z |
| 900719925474099312346 | even | yes | 307.3 | 360 | 32 | 0.4400 | 2026-09-18T12:32:46.781Z |
Machine-readable results: latest.json. Timestamped run directories preserve prior runs, exact inputs, reported probabilities, request IDs, and timings. Reproduce with npm run benchmark after setting TYPESAFE_API_KEY.