f3dx-bench

Experimental, opt-in telemetry pipeline for exploring aggregated LLM request latency, errors, and client-reported cost estimates.

Status

This repository is a prototype, not a live public benchmark or telemetry service. As of 2026-08-10, the configured dashboard and ingest domains are not reachable, and the dashboard's configured parquet object is not publicly readable. Do not send production telemetry to an endpoint inferred from this repository.

There is no client SDK or integration with f3dx or llmkit in this repository. Any such integration would need its own implementation, review, and explicit user opt-in.

What is implemented here

• A versioned JSON Schema for a TraceBeacon request record.
• A Cloudflare Worker prototype with beacon, forget-request, and health routes.
• A Python compaction utility that converts stored NDJSON records to Parquet and can suppress low-count (model, provider, region, hour) buckets when invoked with an appropriate threshold.
• A static dashboard prototype that reads a configured Parquet object in the browser.

The source is an implementation artifact, not evidence that these components are deployed, connected, or receiving real traffic.

Data boundary

The schema intentionally excludes prompt and response content, provider secrets, and HTTP headers. It includes a stable per-install identifier and request metadata such as model, provider, status code, latency, token counts, and an optional client-side cost estimate. Review the exact schema before considering any integration.

The repository contains design notes about privacy, retention, aggregation, and attestation. They describe intended behavior and should not be read as proof of a running service or a completed privacy review.

Repository layout

• schemas/ - beacon wire-format and Parquet layout.
• worker/ - ingest Worker prototype.
• tools/compact_ndjson_to_parquet.py - offline compaction utility.
• dashboard/ - static dashboard prototype.
• docs/ - design notes and open implementation assumptions.

Local checks

The Worker package provides type and lint checks:

cd worker
npm ci
npm run typecheck
npm run lint

These checks validate the Worker source only. They do not validate deployment configuration, data handling in a hosted environment, client opt-in behavior, public aggregation, or the dashboard's availability.

License

MIT.