Moonshine Streaming Tiny
May 16, 2026 · View on GitHub
Useful Sensors' UsefulSensors/moonshine-streaming-tiny
ported to transcribe.cpp. A 34M-parameter encoder-decoder English ASR model
designed for streaming use (ergodic encoder + sliding-window attention,
50 Hz time-domain frontend).
What it's for
Offline English speech-to-text. The model takes a 16 kHz mono WAV and produces a transcript. It does not translate, has no multilingual capability, and does not emit timestamps.
See Useful Sensors' model card for training data, intended use, and upstream evaluation methodology.
Licensed MIT. Ported from upstream commit
f8e9dfd,
pinned 2026-05-06.
Download
| Quantization | Download | Size | WER (LibriSpeech test-clean) |
|---|---|---|---|
| F32 | moonshine-streaming-tiny-F32.gguf | 169 MB | 4.53% |
| F16 | moonshine-streaming-tiny-F16.gguf | 85 MB | 4.53% |
| Q8_0 | moonshine-streaming-tiny-Q8_0.gguf | 48 MB | 4.52% |
WER is measured on the full LibriSpeech test-clean split (2620 utterances)
with greedy decoding (num_beams=1, do_sample=False). F32 reference
baseline: 4.53%. The HF Transformers reference scored on the same manifest
in the same regime lands at 4.52% with 99.6% byte-identical hypotheses to
our F32, so the port is at exact parity with the reference. Useful Sensors'
self-reported number on this split is 4.49% from the Open ASR Leaderboard
table; the +0.04pp residual is a scoring / text-normalization difference vs
that methodology, not a numerical drift in the port.
Q6_K / Q5_K_M / Q4_K_M GGUFs are not currently shipped for this variant.
Streaming vs offline parity (Q8_0)
The streaming session API (transcribe_stream_begin/feed/finalize)
produces a final transcript at parity with the offline one-shot path
on Q8_0:
| Mode | WER | Sub / Del / Ins | CLI errors |
|---|---|---|---|
| Offline (one-shot) | 4.52% | 1764 / 250 / 383 | 0 |
Streaming --stream-chunk-ms 500 | 4.54% | 1772 / 252 / 381 | 0 |
The +0.02 pp delta is 8 extra word errors across ~52 000 reference
words, sitting comfortably inside the 95% confidence interval overlap
([4.22%, 4.88%] vs [4.22%, 4.90%]) and well below any reasonable
regression threshold. The residual comes from PCM trimming during
streaming producing tiny float-precision differences in the encoder
output at the left-context boundary, which occasionally flip a single
argmax late in the AR decode. Reports:
reports/wer/moonshine-streaming-tiny-Q8_0.test-clean.{offline,stream-500ms}.score.json.
Quick Start
cmake -B build
cmake --build build
build/bin/transcribe-cli \
-m models/moonshine-streaming-tiny/moonshine-streaming-tiny-Q8_0.gguf \
samples/jfk.wav
If your audio is not already 16 kHz mono WAV, convert it first:
ffmpeg -i input.mp3 -ar 16000 -ac 1 output.wav
Performance
Cells are wall-clock latency (mean over 5 iterations after 2 warmups),
with speedup over realtime in parentheses. Units: ms below 1 s, s above
(2 decimal places).
Apple M4 Max
| Backend | Sample | Q8_0 |
|---|---|---|
| Metal | jfk (11.0s) | 50 ms (218×) |
| Metal | dots (35.3s) | 355 ms (100×) |
| CPU | jfk (11.0s) | 44 ms (250×) |
| CPU | dots (35.3s) | 206 ms (172×) |
macOS 26.4.1, transcribe.cpp 0d312ce.
AMD Ryzen 7 4750U Pro
| Backend | Sample | Q8_0 |
|---|---|---|
| Vulkan | jfk (11.0s) | 140 ms (79×) |
| Vulkan | dots (35.3s) | 892 ms (40×) |
| CPU | jfk (11.0s) | 160 ms (69×) |
| CPU | dots (35.3s) | 882 ms (40×) |
Fedora 43, transcribe.cpp f243f34. Vulkan device: AMD Radeon Graphics (RADV RENOIR).
Benchmark reproduction:
uv run scripts/bench/run.py \
--models moonshine-streaming-tiny \
--quants q8_0 \
--samples jfk,dots \
--backends metal,cpu,vulkan \
--iters 5 --warmup 2 \
--name moonshine-streaming-publication
Numerical Validation
transcribe.cpp is validated tensor-by-tensor against the HF Transformers
reference (MoonshineStreamingForConditionalGeneration, fp32 inference,
attn_implementation="eager") on samples/jfk.wav. All 37 contract
tensors fall within family tolerance, and the final transcript matches
the reference. Last validated at commit
0d312ce.
| Field | Value |
|---|---|
| Reference | HF Transformers v5.7.0, UsefulSensors/moonshine-streaming-tiny |
| Dump script | scripts/dump_reference_moonshine_streaming_transformers.py |
| Manifest | tests/golden/moonshine_streaming/moonshine-streaming-tiny.manifest.json |
| Command | uv run scripts/validate.py all --family moonshine_streaming --variant moonshine-streaming-tiny |
Selected tensors (max-abs and mean-abs differences, F32 vs reference, on
samples/jfk.wav):
| Tensor | Max abs diff | Mean abs diff | Notes |
|---|---|---|---|
enc.embedder.cmvn.out | 2.240e-04 | 9.942e-06 | Frontend CMVN output |
enc.embedder.linear.out | 2.840e-04 | 6.028e-06 | Time-domain linear projection |
enc.embedder.conv2.out | 9.388e-05 | `1.945e-06$ | \text{After} 2 \times \text{causal} \text{stride}-2 \text{convs} |
| $enc.block.0.out` | 1.099e-03 | 1.158e-05 | First sliding-window attention block |
enc.block.5.out | 6.105e-03 | 2.195e-04 | Last encoder block — depth-amplified BLAS reduction drift |
enc.final | 2.586e-04 | 8.776e-06 | Final encoder LN output |
adapter.out | 2.677e-04 | 9.082e-06 | Encoder→decoder adapter (learned pos-emb add) |
dec.block.0.out | 1.028e-03 | 4.340e-05 | First decoder block |
dec.block.5.out | 2.746e-03 | 2.010e-04 | Last decoder block |
dec.logits_raw.gen20 | 1.829e-03 | 1.186e-04 | Mid-generation logits (token 20) |
The dominant drift source is BLAS reduction-order differences between
PyTorch's matmul kernels and ggml's mul_mat (Accelerate / Metal /
ggml-cpu). Drift accumulates roughly linearly with depth across encoder +
adapter + decoder; the final logit budget stays well below 1e-3 absolute /
1e-4 mean.
Reproduction
Convert
Loads directly from Useful Sensors' Hugging Face repo via
AutoProcessor + MoonshineStreamingForConditionalGeneration. Output path
is derived from the repo id.
uv run --project scripts/envs/moonshine_streaming \
scripts/convert-moonshine_streaming.py UsefulSensors/moonshine-streaming-tiny
Quantize
Run transcribe-quantize once per target quant. Example for F16; repeat with
Q8_0:
build/bin/transcribe-quantize \
models/moonshine-streaming-tiny/moonshine-streaming-tiny-F32.gguf \
models/moonshine-streaming-tiny/moonshine-streaming-tiny-F16.gguf \
--quant F16
Validate
uv run scripts/validate.py all --family moonshine_streaming --variant moonshine-streaming-tiny
WER sweep
uv run scripts/wer/run.py \
--model models/moonshine-streaming-tiny/moonshine-streaming-tiny-F32.gguf \
--manifest samples/wer/test-clean.manifest.jsonl \
--out reports/wer/moonshine-streaming-tiny-F32.test-clean.jsonl
uv run scripts/wer/score.py reports/wer/moonshine-streaming-tiny-F32.test-clean.jsonl