Auto-generated by scripts/gen_benchmark_md.py. Refresh: re-run scripts/benchmark.py (optionally --skip-nemo), then scripts/plot_benchmark.py, then this script.
The same clip fed to parakeet.cpp and to a reference runtime, each pane typing its transcript at the engine's real measured speed, so the faster engine finishes first. parakeet.cpp reproduces NeMo's output byte-for-byte.
parakeet.cpp vs NeMo (PyTorch) on the same GPU (slowed down so the sub-100ms race is watchable):
And on CPU, against NeMo's own PyTorch runtime, about 1.5x faster and still byte-for-byte identical:
Versus whisper.cpp turbo, same accuracy (WER 1.6% on this clip) and far less compute:
- CPU: 20-core host; 8 threads used for both NeMo and parakeet.cpp (matched).
- CPU-only inference throughout — no GPU.
| Component | Version / notes |
|---|
| NeMo | 2.7.3 (PyTorch CPU) |
| parakeet.cpp ggml engine | this repo — GGUF dtypes: f32, f16, q8_0, q6_k, q5_k, q4_k |
| Set | Description |
|---|
| LibriSpeech test-clean | 100 utterances, ~15 min audio; ground-truth transcripts → formal WER |
| Diverse clips | 4 clips (JFK, MLK "I Have a Dream", Italian speech, synthetic TTS) — real-audio sanity check |
- Batch size = 1 for both engines; thread count = 8 (matched).
- NeMo:
torch.set_num_threads(8), per-file timing via time.perf_counter.
- ours:
parakeet-cli bench --threads 8 (load once, time transcribe only); K-quants via parakeet-cli quantize.
- Peak RSS via a
/usr/bin/time -v wrapper.
- RTFx = Σ audio_sec / Σ proc_sec (higher = faster; >1 = real-time capable).
- WER = normalized word error rate vs LibriSpeech ground truth.
- Agreement WER = normalized WER between NeMo output and ours (≈0 ⇒ we reproduce NeMo).
The NeMo numbers are the unchanged PyTorch-CPU reference; only the ggml engine
is re-measured here. f32 output is byte-identical to NeMo (agreement ≈ 0);
lower K-quants trade a little accuracy for size, shown below.
| Model | RTFx NeMo | RTFx f32 | Speedup f32 | RTFx q8_0 | Speedup q8_0 | WER NeMo % | WER f32 % | Agree f32 % | RSS NeMo MB | RSS f32 MB |
|---|
| ctc-0.6b | 30.7 | 41.7 | 1.36× | 49.5 | 1.61× | 4.33 | 4.33 | 0.0000 | 5447 | 2457 |
| ctc-1.1b | 17.8 | 26.1 | 1.46× | 29.0 | 1.63× | 4.73 | 4.75 | 0.0222 | 8914 | 4201 |
| rnnt-0.6b | 24.3 | 34.1 | 1.40× | 37.0 | 1.52× | 4.41 | 4.41 | 0.0000 | 5516 | 2487 |
| rnnt-1.1b | 15.3 | 21.4 | 1.40× | 24.0 | 1.57× | 4.26 | 4.26 | 0.0000 | 8982 | 4231 |
| tdt-0.6b-v2 | 22.4 | 32.4 | 1.45× | 34.7 | 1.55× | 3.71 | 3.71 | 0.0000 | 5499 | 2545 |
| tdt-0.6b-v3 | 21.2 | 29.7 | 1.40× | 34.4 | 1.62× | 3.40 | 3.41 | 0.0143 | 5598 | 2582 |
| tdt-1.1b | 14.6 | 22.6 | 1.54× | 25.5 | 1.74× | 2.51 | 2.51 | 0.0000 | 8956 | 4231 |
| tdt_ctc-1.1b | 13.3 | 22.5 | 1.69× | 25.2 | 1.89× | 2.61 | 2.55 | 0.0985 | 8909 | 4236 |
| tdt_ctc-110m | 72.9 | 81.1 | 1.11× | 91.5 | 1.26× | 3.48 | 3.46 | 0.0196 | 1650 | 563 |
| rt-eou-120m-v1 | 61.8 | 70.6 | 1.14× | 76.1 | 1.23× | 10.92 | 10.92 | 0.0000 | 1714 | 621 |
Speedup = ours RTFx / NeMo RTFx (>1 = faster than NeMo). f32 reproduces NeMo's transcript (agreement ≈ 0).
nemotron-3.5-asr-streaming-0.6b is a FastConformer transducer with a per-language prompt: a one-hot language vector drives a PromptKernel between the encoder and the RNN-T decoder. It runs both offline and cache-aware streaming. Because it loads from a local .nemo plus its GGUF, it sits outside the LibriSpeech pipeline above and is measured on its own here.
One clip (speech.wav, 7.43 s), language prompt en, 8 threads, median of 7 passes after one warmup. ours is parakeet-cli bench --decoder tdt --lang en (load once, time transcribe only); NeMo runs the same prompt forward (preprocessor, encoder, PromptKernel, RNN-T greedy) on PyTorch CPU. RTFx = audio seconds per second of compute; higher is faster.
Host: AMD Ryzen 9 9950X3D (20 cores), CPU-only. NeMo 2.8.0rc0.
| Engine | RTFx | Speedup vs NeMo | Agreement WER vs NeMo |
|---|
| NeMo (PyTorch CPU) | 12.2 | 1.00× | reference |
| parakeet.cpp f32 | 29.4 | 2.40× | 0.0000% |
| parakeet.cpp q8_0 | 30.8 | 2.52× | 0.0000% |
Accuracy is WER 0 vs NeMo: the f32 and q8_0 transcripts are byte-identical to NeMo's on the timed runs (agreement WER 0.0000%), so the speed numbers compare equal work. parakeet.cpp is 2.40× faster than NeMo at f32 and 2.52× at q8_0.
On the GPU (NVIDIA GB10, same single clip and 7-pass median, both engines on the device), parakeet.cpp still wins, with a smaller margin than on CPU. NeMo runs natively on the GB10 here via torch 2.11 + cu128 (no nvcr container needed), and its RNN-T greedy decode is CUDA-graph accelerated, so the gap narrows. The larger GPU wins in this repo come from the TDT and hybrid models, where NeMo falls back to a slower per-step Python loop; nemotron is RNN-T, so it does not hit that path.
Host: NVIDIA GB10 Grace-Blackwell (sm_121), CUDA 13. NeMo main, torch 2.11.0+cu128.
| Engine | RTFx | Speedup vs NeMo | Agreement WER vs NeMo |
|---|
| NeMo (PyTorch GPU) | 91.8 | 1.00× | reference |
| parakeet.cpp f32 | 106.5 | 1.16× | 0.0000% |
| parakeet.cpp q8_0 | 119.8 | 1.30× | 0.0000% |
The transcripts are byte-identical to NeMo's on GPU as well (WER 0).
Streaming path (f32, cache-aware): compute RTFx 3.80 (median wall 2503 ms over the 7.43 s clip, one-time model load of 548 ms subtracted). Streaming is latency-oriented: it runs many small chunked forward passes rather than one offline pass, so its RTFx sits well below the offline number by design while staying several times real time. The streaming transcript matches the offline and NeMo transcripts.
Averaged over all models (LibriSpeech). Size is the mean GGUF size as a fraction of the f32 GGUF.
| dtype | avg size vs f32 | mean speedup vs NeMo | mean WER vs truth % | mean agreement vs NeMo % |
|---|
| f32 | 100% | 1.40× | 4.43 | 0.015 |
| f16 | 57% | 1.70× | 4.43 | 0.015 |
| q8_0 | 37% | 1.56× | 4.47 | 0.160 |
| q6_k | 32% | 1.27× | 4.41 | 0.438 |
| q5_k | 29% | 1.05× | 4.42 | 0.451 |
| q4_k | 26% | 1.25× | 4.21 | 1.084 |
f32 is the faithful reference (agreement ≈ 0). q8_0 is near-lossless; K-quants (q6_k→q4_k) shrink the model further at a small, monotonic accuracy cost. See the per-model quant plots below.
Decode batching coalesces the per-step prediction-LSTM and joint-network GEMMs across several utterances into single batched ops, so one decode loop advances B clips at once. It applies to transducer (TDT/RNN-T) models only — CTC has no autoregressive decode to batch. Speedup is serial_ms / batched_ms: the wall-clock of B independent single-clip decodes divided by one batched decode over the same B copies (encoder cost is paid once and excluded). The clips/s @B=16 column is the batched decode throughput at B=16.
CPU (cpu, q5_k, 8 threads), best-of-5, one clip replicated B times.
| Model | B=1 | B=4 | B=8 | B=16 | clips/s @B=16 |
|---|
| rnnt-0.6b | 0.98× | 2.81× | 3.91× | 5.01× | 378.9 |
| rnnt-1.1b | 1.01× | 2.27× | 3.54× | 4.19× | 366.1 |
| rt-eou-120m-v1 | 1.00× | 2.32× | 3.00× | 3.36× | 638.1 |
| tdt-0.6b-v2 | 0.90× | 2.78× | 4.12× | 5.19× | 458.8 |
| tdt-0.6b-v3 | 0.82× | 3.26× | 5.90× | 4.88× | 253.2 |
| tdt-1.1b | 0.98× | 2.55× | 3.74× | 4.57× | 483.2 |
| tdt_ctc-1.1b | 1.00× | 2.59× | 3.80× | 4.73× | 458.8 |
| tdt_ctc-110m | 1.03× | 2.46× | 3.12× | 3.61× | 834.5 |
GPU (CUDA0, f16, 8 threads), best-of-5, one clip replicated B times.
| Model | B=1 | B=4 | B=8 | B=16 | clips/s @B=16 |
|---|
| rnnt-0.6b | 1.00× | 3.32× | 6.46× | 11.44× | 585.8 |
| rnnt-1.1b | 1.00× | 3.31× | 6.30× | 11.17× | 594.1 |
| rt-eou-120m-v1 | 0.98× | 3.11× | 5.87× | 10.25× | 951.1 |
| tdt-0.6b-v2 | 1.00× | 3.61× | 6.88× | 12.42× | 712.0 |
| tdt-0.6b-v3 | 1.00× | 3.45× | 6.49× | 11.45× | 614.5 |
| tdt-1.1b | 1.00× | 3.55× | 6.72× | 12.17× | 817.9 |
| tdt_ctc-1.1b | 1.00× | 3.45× | 6.50× | 11.47× | 746.9 |
| tdt_ctc-110m | 1.01× | 3.29× | 6.33× | 10.89× | 1259.7 |
parakeet.cpp runs on Apple GPUs through ggml's Metal backend
(-DPARAKEET_GGML_METAL=ON). The encoder runs on the GPU; ops without a Metal
kernel fall back to CPU automatically, so any model runs.
Apple M4, macOS 26.5. q4_k weights, one 7.4 s clip, best-of-6 steady-state runs
(model load and one-time pipeline compilation excluded), via parakeet-cli bench.
RTFx is audio seconds per second of compute; higher is faster.
| Model | Metal | CPU | Speedup |
|---|
| ctc-0.6b | 91.1 | 16.9 | 5.4× |
| ctc-1.1b | 56.2 | 10.0 | 5.6× |
| parakeet-tdt_ctc-110m | 99.4 | 65.5 | 1.5× |
| realtime_eou_120m-v1 | 76.5 | 58.6 | 1.3× |
| rnnt-0.6b | 53.0 | 20.3 | 2.6× |
| rnnt-1.1b | 39.4 | 12.4 | 3.2× |
| tdt-0.6b-v2 | 57.1 | 20.1 | 2.8× |
| tdt-0.6b-v3 | 54.3 | 19.8 | 2.7× |
| tdt-1.1b | 43.8 | 12.6 | 3.5× |
| tdt_ctc-110m | 103.1 | 65.9 | 1.6× |
| tdt_ctc-1.1b | 41.9 | 12.5 | 3.4× |
Metal helps most on the larger models (about 3× to 5×). The 110m and 120m models
stay closer to CPU because their short encoders are dominated by per-run fixed
costs. Two ops the FastConformer encoder needs ship as native Metal kernels here:
the subsampling depthwise conv (CONV_2D_DW) and the conv module's leading-side
time padding (PAD); with these the encoder layers run entirely on the GPU. The
only work left on the CPU is the one-time log-mel front end: its ops all have
Metal kernels, but the scheduler keeps that small, once-per-clip subgraph (a
couple of matmuls plus a log over host-resident inputs) on the CPU rather than
uploading it, which for its size is the faster choice. Reproduce with
scripts/bench_metal_dw.sh <model.gguf> <clip.wav>.
Transcripts from the diverse clip set (no ground truth for most). NeMo vs parakeet.cpp f32 side-by-side to confirm fidelity on real audio.
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | first of all conicera one of the primi video go fato probably me the primo shorto oil second deal terzo suquestogan le era proposido de l rangeros si mr farmer product thequesto agric toore sicidiano malto giovan maltoedalista you know nongosco personand dea odejiso pradigaad conz yarvero pera secondavolt time for my un popcular peto alltravolta aloaltro vistoro l rangea cbra n logonosco andon gem motivo di publigitsare suway produot n non aruno sponsor in guestoganale percevacho pububigita gratuita lange perke credo okay in nostrog bos has sebrepeggiore e la grande distributione cosic ormaangegeiic fruto verdura vano combra la merch al i grandi mercati orto frutioido citrova callitas |
| parakeet.cpp f32 | first of all conicera one of the primi video go fato probably me the primo shorto oil second deal terzo suquestogan le era proposido de l rangeros si mr farmer product thequesto agric toore sicidiano malto giovan maltoedalista you know nongosco personand dea odejiso pradigaad conz yarvero pera secondavolt time for my un popcular peto alltravolta aloaltro vistoro l rangea cbra n logonosco andon gem motivo di publigitsare suway produot n non aruno sponsor in guestoganale percevacho pububigita gratuita lange perke credo okay in nostrog bos has sebrepeggiore e la grande distributione cosic ormaangegeiic fruto verdura vano combra la merch al i grandi mercati orto frutioido citrova callitas |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | i have the pleasure to present to you dr martin luther king i am happy to jon with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation five score years ago a great american in whose symbolic shadow we stand today signed the emancipation proclamation this momentous decree came as a great beacon light of hope to millions of negro slaves who had been seared in the flames of withering injustice it came as a joyous daybreak to end the long night of their captivity but one hundred years later the negro still is not free one hundred years later the life of the negro is still sadly crippled by the manacles of segregation and the chains of discrimination one hundred years later the negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity one hundred years later the negro is still languished in the corners of american society and finds himself in exile in his own land and so we've come here today to dramatize a shameful condition innocence we've come to our nation's capital to cash a check when the architects of our republic wrote the magnificent words of the constitution and the declaration of independence they were signing a promissory not to which every american was to fall this note was a promise that all men yes black men as well as white men would bearanted |
| parakeet.cpp f32 | i have the pleasure to present to you dr martin luther king i am happy to jon with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation five score years ago a great american in whose symbolic shadow we stand today signed the emancipation proclamation this momentous decree came as a great beacon light of hope to millions of negro slaves who had been seared in the flames of withering injustice it came as a joyous daybreak to end the long night of their captivity but one hundred years later the negro still is not free one hundred years later the life of the negro is still sadly crippled by the manacles of segregation and the chains of discrimination one hundred years later the negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity one hundred years later the negro is still languished in the corners of american society and finds himself in exile in his own land and so we've come here today to dramatize a shameful condition innocence we've come to our nation's capital to cash a check when the architects of our republic wrote the magnificent words of the constitution and the declaration of independence they were signing a promissory not to which every american was to fall this note was a promise that all men yes black men as well as white men would be gar |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | and so my fellow americans ask not what your country can do for you ask what you can do for your country |
| parakeet.cpp f32 | and so my fellow americans ask not what your country can do for you ask what you can do for your country |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | hello this is a test of the voxtrol speech to text system |
| parakeet.cpp f32 | hello this is a test of the voxtrol speech to text system |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | first of all couldaorder i one of the primie video fat probably went the primo short oil second or third so so cestogana leera proposito de langeros the mr farmer producted a questesto agricultore sigidiano malto giovan maltoedalista you nonosco person and mendema o degiso pic anda d conziar velo perla second of all time for unpopu car respect to ultravolta alora traaltro vistoroa rangeaombro nol loconosco and anjem motivo d publigitale is who i product yeah and an aruno sponsor in guestoanale perquvachcio buligita gratuita l range perqu grarado okay inllstrogios has andrepeggiore la grande distributiona cicom or may anpically fr verdura van no combra la merch al i grandi mercati oro fruit t callido citrova collitas |
| parakeet.cpp f32 | first of all couldaorder i one of the primie video fat probably went the primo short oil second or third so so cestogana leera proposito de langeros the mr farmer producted a questesto agricultore sigidiano malto giovan maltoedalista you nonosco person and mendema o degiso pic anda d conziar velo perla second of all time for unpopu car respect to ultravolta alora traaltro vistoroa rangeaombro nol loconosco and anjem motivo d publigitale is who i product yeah and an aruno sponsor in guestoanale perquvachcio buligita gratuita l range perqu grarado okay inllstrogios has andrepeggiore la grande distributiona cicom or may anpically fr verdura van no combra la merch al i grandi mercati oro fruit t callido citrova collitas |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | i have the pleasure to present to you dr martin luther king i am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation five score years ago a great american in whose symbolic shadow we stand today signed the emancipation proclamation this momentous decree came as a great beacon light of hope to millions of negro slaves who had been seared in the flames of withering injustice it came as a joyous daybreak to end the long night of their captivity but one hundred years later the negro still is not free one hundred years later the life of the negro is still sadly crippled by the manacles of segregation and the chains of discrimination one hundred years later the negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity one hundred years later the negro has still languished in the corners of american society and finds himself in exile in his own land and so we've come here today to dramatize a shameful condition in a sense we've come to our nation's capital to cash a check when the architects of our republic wrote the magnificent words of the constitution and the declaration of independence they were signing a promissory note to which every american was to fall heir this note was a promise that all men yes black men as well as white men would be guaranteed |
| parakeet.cpp f32 | i have the pleasure to present to you dr martin luther king i am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation five score years ago a great american in whose symbolic shadow we stand today signed the emancipation proclamation this momentous decree came as a great beacon light of hope to millions of negro slaves who had been seared in the flames of withering injustice it came as a joyous daybreak to end the long night of their captivity but one hundred years later the negro still is not free one hundred years later the life of the negro is still sadly crippled by the manacles of segregation and the chains of discrimination one hundred years later the negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity one hundred years later the negro has still languished in the corners of american society and finds himself in exile in his own land and so we've come here today to dramatize a shameful condition in a sense we've come to our nation's capital to cash a check when the architects of our republic wrote the magnificent words of the constitution and the declaration of independence they were signing a promissory note to which every american was to fall heir this note was a promise that all men yes black men as well as white men would be guaranteed |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | and so my fellow americans ask not what your country can do for you ask what you can do for your country |
| parakeet.cpp f32 | and so my fellow americans ask not what your country can do for you ask what you can do for your country |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | hello this is a test of the voxtstrol speech to text system |
| parakeet.cpp f32 | hello this is a test of the voxtstrol speech to text system |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | force primivido fato probably primo shortoil secondal terzo suquestogana le era proposito de la rangero cedi mister farmer producted a questo gricultore sicidiano molto giovane molto idalista non gonosco persona mendema odegiso predicanda dicon zi yarverlo per la seconda voltai informa unbopucarispe volta altro vistriole range motivo di published ari sui producu noncuno sponsor in questoganale perkevacu publishta graduate a lerange perque credo que inostruga sembrepeggiore e le grande distribution verduva no combra la merch alla i grandi mercati or to fruiti coridova sitrova qualitas |
| parakeet.cpp f32 | force primivido fato probably primo shortoil secondal terzo suquestogana le era proposito de la rangero cedi mister farmer producted a questo gricultore sicidiano molto giovane molto idalista non gonosco persona mendema odegiso predicanda dicon zi yarverlo per la seconda voltai informa unbopucarispe volta altro vistriole range motivo di published ari sui producu noncuno sponsor in questoganale perkevacu publishta graduate a lerange perque credo que inostruga sembrepeggiore e le grande distribution verduva no combra la merch alla i grandi mercati or to fruiti coridova sitrova qualitas |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | i have the pleasure to present to you dr martin luther king i am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation five score years ago a great american in whose symbolic shadow we stand today signed the emancipation proclamation this momentous decree came as a great beacon light of hope to millions of negro slaves who had been seared in the flames of withering injustice it came as a joyous daybreak to end the long night of their captivity but one hundred years later the negro still is not free one hundred years later the life of the negro is still sadly crippled by the manacles of segregation and the chains of discrimination one hundred years later the negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity one hundred years later the negro is still languished in the corners of american society and finds himself in exile in his own land so we've come here today to dramatize a shameful condition in a sense we've come to our nation's capital to cash a check when the architects of our republic wrote the magnificent words of the constitution and the declaration of independence they were signing a promissory note to whichever american was to fall this note was a promise that all men yes black men as well as white men would be guaranteed |
| parakeet.cpp f32 | i have the pleasure to present to you dr martin luther king i am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation five score years ago a great american in whose symbolic shadow we stand today signed the emancipation proclamation this momentous decree came as a great beacon light of hope to millions of negro slaves who had been seared in the flames of withering injustice it came as a joyous daybreak to end the long night of their captivity but one hundred years later the negro still is not free one hundred years later the life of the negro is still sadly crippled by the manacles of segregation and the chains of discrimination one hundred years later the negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity one hundred years later the negro is still languished in the corners of american society and finds himself in exile in his own land so we've come here today to dramatize a shameful condition in a sense we've come to our nation's capital to cash a check when the architects of our republic wrote the magnificent words of the constitution and the declaration of independence they were signing a promissory note to whichever american was to fall this note was a promise that all men yes black men as well as white men would be guaranteed |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | and so my fellow americans ask not what your country can do for you ask what you can do for your country |
| parakeet.cpp f32 | and so my fellow americans ask not what your country can do for you ask what you can do for your country |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | hello this is a test of the voxtrill speech to text system |
| parakeet.cpp f32 | hello this is a test of the vauxtroll speech to text system |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | forseucono rigordera que uno dei primi video queo fato probabenti primo shorto il secondal terzo suquesto canale era proposido della rangerosse di mister farmer produceda questo agricultore sicidiano molto giovane molto i delista que io non gonosco persona mendema o degiso prodiganda di con sigarvelo per la seconda volta informa un popucciare espevolta alora ter altro vistorio la rangeare combro nolo conosco en ungem motivo di publigitare y suae produdottie non vaccio non arcuno sponsor inquesto canale perque vaccio publigita gratuita le range per que credo que in nostrogi posia sembre peggiore e la grande distribuzione cosi com ermur mayang epiculi fruito verdurava no cumbra i la merche alla ai grandi mercati orto fruti corid si trova qualitas |
| parakeet.cpp f32 | forseucono rigordera que uno dei primi video queo fato probabenti primo shorto il secondal terzo suquesto canale era proposido della rangerosse di mister farmer produceda questo agricultore sicidiano molto giovane molto i delista que io non gonosco persona mendema o degiso prodiganda di con sigarvelo per la seconda volta informa un popucciare espevolta alora ter altro vistorio la rangeare combro nolo conosco en ungem motivo di publigitare y suae produdottie non vaccio non arcuno sponsor inquesto canale perque vaccio publigita gratuita le range per que credo que in nostrogi posia sembre peggiore e la grande distribuzione cosi com ermur mayang epiculi fruito verdurava no cumbra i la merche alla ai grandi mercati orto fruti corid si trova qualitas |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | i have the pleasure to present to you dr martin luther king i am happy to john with you today in history as the greatest demonstration freedom in the history of our nation five years ago a great american in whose symbolic shadow we stand today the emancipation proclamation this momentous decree came as a grand beacon light of hope to millions of negro slaves who had been seared in the flames of withering injustice it came as a joyous daybreak to end the long night of their captivity but one hundred years later the negro still is not free one hundred years later the life of the negro is still sadly crippled by the manacles of segregation and the chains of discrimination one hundred years later the negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity one hundred years later the negro is still languished in the corners of american society and finds himself in exile in his own land so we've come here today to dramatize a shameful condition in a sense we've come to our nation's capital to cash a check when the architects of our republic wrote the magnificent words of the constitution and the declaration of independence they was signing a promise or a note to which every american was to fall this note was a promise that all men yes black men as well as white men would be guaranteed |
| parakeet.cpp f32 | i have the pleasure to present to you dr martin luther king i am happy to john with you today in history as the greatest demonstration freedom in the history of our nation five years ago a great american in whose symbolic shadow we stand today the emancipation proclamation this momentous decree came as a grand beacon light of hope to millions of negro slaves who had been seared in the flames of withering injustice it came as a joyous daybreak to end the long night of their captivity but one hundred years later the negro still is not free one hundred years later the life of the negro is still sadly crippled by the manacles of segregation and the chains of discrimination one hundred years later the negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity one hundred years later the negro is still languished in the corners of american society and finds himself in exile in his own land so we've come here today to dramatize a shameful condition in a sense we've come to our nation's capital to cash a check when the architects of our republic wrote the magnificent words of the constitution and the declaration of independence they was signing a promise or a note to which every american was to fall this note was a promise that all men yes black men as well as white men would be guaranteed |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | and so my fellow americans ask not what your country can do for you ask what you can do for your country |
| parakeet.cpp f32 | and so my fellow americans ask not what your country can do for you ask what you can do for your country |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | hello this is a test of the voxtroll speech to text system |
| parakeet.cpp f32 | hello this is a test of the voxtroll speech to text system |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | For some of them that one of the first videos that I first short or second, this proposition of the range of Mr. Farmer products of this agriculture, very idealist that people, but the second respect of the Volta. So, vista range, not with, in a motive, public public and their product, not a sponsor in this sponsor, because it will publish the range. Because the book is very distribution, as well as the verdict, the merch to fruit or two, if you have quality. |
| parakeet.cpp f32 | For some of them that one of the first videos that I first short or second, this proposition of the range of Mr. Farmer products of this agriculture, very idealist that people, but the second respect of the Volta. So, vista range, not with, in a motive, public public and their product, not a sponsor in this sponsor, because it will publish the range. Because the book is very distribution, as well as the verdict, the merch to fruit or two, if you have quality. |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | I have the pleasure to present to you Dr. Martin Luther King, DR. I am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation. Five score years ago, a great American in whose symbolic shadow we stand today signed the Emancipation Proclamation. This momentous decree came as a great beacon light of hope to millions of Negro slaves who had been seared in the flames of withering injustice. It came as a joyous daybreak to end the long night of their captivity. But 100 years later, the Negro still is not free. 100 years later, the life of the Negro is still sadly crippled by the manacles of segregation and the chains of discrimination. 100 years later, the Negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity. 100 years later, the Negro is still languished in the corners of American society and finds himself in exile in his own land. And so we've come here today to dramatize a shameful condition. In a sense, we've come to our nation's capital to cash a check. When the architects of our republic wrote the magnificent words of the Constitution and the Declaration of Independence, they were signing a promissory note to which every American was to fall heir. This note was a promise that all men, yes, black men as well as white men, would be guaranteed. |
| parakeet.cpp f32 | I have the pleasure to present to you Dr. Martin Luther King, DR. I am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation. Five score years ago, a great American in whose symbolic shadow we stand today signed the Emancipation Proclamation. This momentous decree came as a great beacon light of hope to millions of Negro slaves who had been seared in the flames of withering injustice. It came as a joyous daybreak to end the long night of their captivity. But 100 years later, the Negro still is not free. 100 years later, the life of the Negro is still sadly crippled by the manacles of segregation and the chains of discrimination. 100 years later, the Negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity. 100 years later, the Negro is still languished in the corners of American society and finds himself in exile in his own land. And so we've come here today to dramatize a shameful condition. In a sense, we've come to our nation's capital to cash a check. When the architects of our republic wrote the magnificent words of the Constitution and the Declaration of Independence, they were signing a promissory note to which every American was to fall heir. This note was a promise that all men, yes, black men as well as white men, would be guaranteed. |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. |
| parakeet.cpp f32 | And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | Hello, this is a test of the Voxtral speech-to-text system. |
| parakeet.cpp f32 | Hello, this is a test of the Voxtral speech-to-text system. |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | Forse qualcuno ricorderà che uno dei primi video che ho fatto, probabilmente il primo short o il secondo, il terzo su questo canale era a proposito delle arance rosse di Mr. Farmer prodotte da questo agricoltore siciliano molto giovane, molto idealista che io non conosco personalmente, ma ho deciso praticamente di consigliarvelo per la seconda volta in forma un po' più chiara rispetto all'altra volta. Allora, tra l'altro, visto che io le arance le compro, non lo conosco e non c'è motivo di pubblicizzare i suoi prodotti e non ho alcuno sponsor in questo canale perché faccio pubblicità gratuita all'arange perché credo che il nostro cibo sia sempre peggiore e la grande distribuzione, così come ormai anche piccoli frutta di verdura vanno a comprare la merce ai grandi mercati ortofrutticoli dove si trova qualità scarica. |
| parakeet.cpp f32 | Forse qualcuno ricorderà che uno dei primi video che ho fatto, probabilmente il primo short o il secondo, il terzo su questo canale era a proposito delle arance rosse di Mr. Farmer prodotte da questo agricoltore siciliano molto giovane, molto idealista che io non conosco personalmente, ma ho deciso praticamente di consigliarvelo per la seconda volta in forma un po' più chiara rispetto all'altra volta. Allora, tra l'altro, visto che io le arance le compro, non lo conosco e non c'è motivo di pubblicizzare i suoi prodotti e non ho alcuno sponsor in questo canale perché faccio pubblicità gratuita all'arange perché credo che il nostro cibo sia sempre peggiore e la grande distribuzione, così come ormai anche piccoli frutta di verdura vanno a comprare la merce ai grandi mercati ortofrutticoli dove si trova qualità scarica. |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | I have the pleasure to present to you Dr. Martin Luther King. I am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation. Five score years ago, a great American in whose symbolic shadow we stand today signed the Emancipation Proclamation. This momentous decree came as a great beacon light of hope to millions of Negro slaves who had been seared in the flames of withering injustice. It came as a joyous daybreak to end the long night of their captivity. But one hundred years later, the Negro still is not free. One hundred years later, the life of the Negro is still sadly crippled by the manacles of segregation and the chains of discrimination. One hundred years later. In the midst of a vast ocean of material prosperity, one hundred years later. And so we've come here today to dramatize a shameful condition. In a sense, we've come to our nation's capital to cash a check. When the architects of our republic wrote the magnificent words of the Constitution and the Declaration of Independence, they were signing a promissory note to which every American was to fall air. This note was a promise that all men, yes, black men as well as white men would be guaranteed. |
| parakeet.cpp f32 | I have the pleasure to present to you Dr. Martin Luther King. I am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation. Five score years ago, a great American in whose symbolic shadow we stand today signed the Emancipation Proclamation. This momentous decree came as a great beacon light of hope to millions of Negro slaves who had been seared in the flames of withering injustice. It came as a joyous daybreak to end the long night of their captivity. But one hundred years later, the Negro still is not free. One hundred years later, the life of the Negro is still sadly crippled by the manacles of segregation and the chains of discrimination. One hundred years later. In the midst of a vast ocean of material prosperity, one hundred years later. And so we've come here today to dramatize a shameful condition. In a sense, we've come to our nation's capital to cash a check. When the architects of our republic wrote the magnificent words of the Constitution and the Declaration of Independence, they were signing a promissory note to which every American was to fall air. This note was a promise that all men, yes, black men as well as white men would be guaranteed. |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. |
| parakeet.cpp f32 | And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country. |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | Hello, this is a test of the Voxtrel speech to text system. |
| parakeet.cpp f32 | Hello, this is a test of the Voxtrel speech to text system. |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | forse ocono ricordera queuno de primi video queo fato probabe primo shorto il secondl terzo suquestocanale era proposido del rangeerosse di mister farmer prodoteda questo agricultore sigidiano molto giovane molto i dealista que i non gonosco persona mendema o degiso pro diganda di con zi garvelo per la seconda volta informa un popucarres peto la'ravolta alora terraltro vistorio le rangea de combro nolo conosco engemotivo di publigizare suai prodottia non vaccio non arcono sponsor inquesto canale per que vaccio publigita gratuita le range per que credo que in ostrogi bosia sembrepeggiore e la grande distribuzione cosicoma ange picoli fruito verduravano cumbra i la merche alla ai grandi mercati orto fruti co i dove sitrova qualitas |
| parakeet.cpp f32 | forse ocono ricordera queuno de primi video queo fato probabe primo shorto il secondl terzo suquestocanale era proposido del rangeerosse di mister farmer prodoteda questo agricultore sigidiano molto giovane molto i dealista que i non gonosco persona mendema o degiso pro diganda di con zi garvelo per la seconda volta informa un popucarres peto la'ravolta alora terraltro vistorio le rangea de combro nolo conosco engemotivo di publigizare suai prodottia non vaccio non arcono sponsor inquesto canale per que vaccio publigita gratuita le range per que credo que in ostrogi bosia sembrepeggiore e la grande distribuzione cosicoma ange picoli fruito verduravano cumbra i la merche alla ai grandi mercati orto fruti co i dove sitrova qualitas |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | i have the pleasure to present to you dr martin luther king i am happy to john with you today and what will go down in history as the greatest demonstration for freedom in the history of our nation five score years ago a great american in whose symbolic shadow we stand today sign of the emancipation proclamation this momentous decree came as a grand beacon light of hope to millions of negro slaves who had been seared in the flames of withering injustice it came as a joyous daybreak to end the long night of their captivity but one hundred years later the negro still is not free one hundred years later the life of the negro is still sadly crippled by the manacles of segregation and the chains of discrimination one hundred years later the negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity one hundred years later the negro is still lying wished in the corners of american society and finds himself in exile in his own land and so we've come here today to dramatize a shameful condition in a sense we've come to our nation's capital to cash a check when the architects of our republic wrote the magnificent words of the constitution and the declaration of independence they were signing a promissory note to which every american was to fall and this note was a promise that all men yes black men as well as white men would be |
| parakeet.cpp f32 | i have the pleasure to present to you dr martin luther king i am happy to john with you today and what will go down in history as the greatest demonstration for freedom in the history of our nation five score years ago a great american in whose symbolic shadow we stand today sign of the emancipation proclamation this momentous decree came as a grand beacon light of hope to millions of negro slaves who had been seared in the flames of withering injustice it came as a joyous daybreak to end the long night of their captivity but one hundred years later the negro still is not free one hundred years later the life of the negro is still sadly crippled by the manacles of segregation and the chains of discrimination one hundred years later the negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity one hundred years later the negro is still lying wished in the corners of american society and finds himself in exile in his own land and so we've come here today to dramatize a shameful condition in a sense we've come to our nation's capital to cash a check when the architects of our republic wrote the magnificent words of the constitution and the declaration of independence they were signing a promissory note to which every american was to fall and this note was a promise that all men yes black men as well as white men would be |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | and so my fellow americans ask not what your country can do for you ask what you can do for your country |
| parakeet.cpp f32 | and so my fellow americans ask not what your country can do for you ask what you can do for your country |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | hello this is a test of the voxtroll speech to text system |
| parakeet.cpp f32 | hello this is a test of the voxtroll speech to text system |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | del terzo. So cuesto cana leer proposido delleer angero sedi mister Farmer Prodote da cuesto agricultore sigido molto giovane molto edelista que eon gonos co persona mendema o degiso radicandi con igiarlo perla secunda volta informa unpopuque respecto Laltravolta Altra Altro vistoriola Rangeli Compro nolo conosco and gem motivo di pubigitzare prodi and arcuno sponsor in Guesto Canale Perqueciopita Gratuita Le Range Perque Crado Que in losiposias brapeiore e le Grande distribution equos coma pico verdur bano combra la merch a la grandi mercati or to frutiquolidov situ task. |
| parakeet.cpp f32 | For selcungor de la que uno de primi vido que fatovende primo shortoil secured del terzo. So cuesto cana leera proposido deli rangeros se di mister Farmer Prodot de la cuesto agricultore sigido molto giovane molto edel ista que yon gonosco persona o degiso radicandi conzigarverlo per la seconda volta informa unpopuque respecto laltravolta Alora Traaltro vistoriole rangely combro nolo conosco and ungemotivo di publigare i huy prodi and arcunosponsor in Questo Canale Perque pacho pubigita gratuita le range perque crado que in los Ruchiposias enrepeggiore e le Grande distribution equosic coma ange picculi fruitoverdura van combra la merch a la grandi mercati orto fruticolidov situ qualitas. |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | I have the pleasure to present to you Dr. Martin Luther King De Art I am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation Fivescore years ago a great American in whose symbolic shadow we stand today signed the Emancipation Proclamation. This momentous decree came as a great beacon light of hope to millions of Negro slaves who had been seared in the flames of withering injustice. It came as a joyous daybreak to end the long night of their captivity. But one hundred years later the Negro still is not free. one hundred years later the life of the Negro is still sadly crippled by the manacles of segregation and the chains of discrimination one hundred years later. The Negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity one hundred years later the negro is still languished in the corners of American society and finds himself in exile in his own land. And so we've come here today to dramatize a shameful condition. In a sense we've come to our nation's capital to cash a check when the architects of our Republic wrote the magnificent words of the Constitution and the Declaration of Independence. They were signing a promissory note to which every American was to fall heir. This note was a promise that all men yes, black men as well as white men would be guaranteed. |
| parakeet.cpp f32 | I have the pleasure to present to you Dr. Martin Luther King Dear I am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation Fivescore years ago, a great American in whose symbolic shadow we stand today signed the Emancipation Proclamation. This momentous decree came as a great beacon light of hope to millions of Negro slaves who had been seared in the flames of withering injustice. It came as a joyous daybreak to end the long night of their captivity. But one hundred years later, the Negro still is not free. one hundred years later, the life of the Negro is still sadly crippled by the manacles of segregation and the chains of discrimination one hundred years later. The Negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity, one hundred years later, the Negro is still languished in the corners of American society and finds himself in exile in his own land. And so we've come here today to dramatize a shameful condition. In a sense, we've come to our nation's capital to cash a check. When the architects of our Republic wrote the magnificent words of the Constitution and the Declaration of Independence, they were signing a promissory note to which every American was to fall heir. This note was a promise that all men, yes, black men as well as white men would be guaranteed. |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country. |
| parakeet.cpp f32 | And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country. |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | Hello, this is a test of the VoxDrill speech to text system. |
| parakeet.cpp f32 | Hello, this is a test of the VoxDrill speech to text system. |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | First of all Gordera Kuno de Prime video fato probably meant that Primo Shortoil Secundel Terzo Sukestogana Proposito del Ranger Osedi mister Farmer Prodote Aquesto Grigoltore Sigidiano Malto Giovanni, Malta Edelista Non Gonosco persona Mendema Odizo Protica and the Decon Zarvello Perla Second of all time forma Umbopurespecto altravolta Alo Teraltro Vistoriol Ranger Gombroganosco and Jem Motivodi Publigare is Wiperoti and Volcano sponsoring Guestoanale Perkavita Graduita Le Range Perque in Lostrogas and Brepeor Le Grande Distribution Cosicomic Verdura Vanocombre Lamerche Al Grandi Mercati or To Fruit Colido Citrov Koleitas |
| parakeet.cpp f32 | First of all Gordera Kuno de Prime video fato probably meant that Primo Shortoil Secundel Terzo Sukestogana Proposito del Ranger Osedi mister Farmer Prodote Aquesto Grigoltore Sigidiano Malto Giovanni, Malta Edelista Non Gonosco persona Mendema Odizo Protica and the Decon Zarvello Perla Second of all time forma Umbopurespecto altravolta Alo Teraltro Vistoriol Ranger Gombroganosco and Jem Motivodi Publigare is Wiperoti and Volcano sponsoring Guestoanale Perkavita Graduita Le Range Perque in Lostrogas and Brepeor Le Grande Distribution Cosicomic Verdura Vanocombre Lamerche Al Grandi Mercati or To Fruit Colido Citrov Koleitas |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | I have the pleasure to present to you doctor Martin Luther King John. I am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation five score years ago a great American in whose symbolic shadow we stand today signed the emancipation proclamation this momentous decree came as a great beacon light of hope to millions of negro slaves who had been seared in the flames of withering injustice. It came as a joyous daybreak to end the long night of their captivity but one hundred years later the negro still is not free one hundred years later the life of the negro is still sadly crippled by the manacles of segregation and the chains of discrimination one hundred years later the negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity one hundred years later the negro is still languished in the corners of American society and finds himself in exile in his own land and so we've come here today to dramatize a shameful condition in a sense we've come to our nation's capital to cash a check when the architects of our Republic wrote the magnificent words of the Constitution and the Declaration of Independence they were signing a promisory note to whichever American was to fall out. This note was a promise at all men yes black men as well as white men would be guaranteed. |
| parakeet.cpp f32 | I have the pleasure to present to you doctor Martin Luther King John. I am happy to join with you today in what will go down in history as the greatest demonstration for freedom in the history of our nation five score years ago a great American in whose symbolic shadow we stand today signed the emancipation proclamation this momentous decree came as a great beacon light of hope to millions of negro slaves who had been seared in the flames of withering injustice. It came as a joyous daybreak to end the long night of their captivity but one hundred years later the negro still is not free one hundred years later the life of the negro is still sadly crippled by the manacles of segregation and the chains of discrimination one hundred years later the negro lives on a lonely island of poverty in the midst of a vast ocean of material prosperity one hundred years later the negro is still languished in the corners of American society and finds himself in exile in his own land and so we've come here today to dramatize a shameful condition in a sense we've come to our nation's capital to cash a check when the architects of our Republic wrote the magnificent words of the Constitution and the Declaration of Independence they were signing a promisory note to whichever American was to fall out. This note was a promise at all men yes black men as well as white men would be guaranteed. |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | And so my fellow Americans, ask not what your country can do for you. Ask what you can do for your country. |
| parakeet.cpp f32 | And so my fellow Americans, ask not what your country can do for you. Ask what you can do for your country. |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | Hello, this is a test of the VoxTral speech to text system. |
| parakeet.cpp f32 | Hello, this is a test of the VoxTral speech to text system. |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | for kuna you go there probably when they primo shortly multidalista que personal respectively mercadi or to fruit |
| parakeet.cpp f32 | for kuna you go there probably when they primo shortly multidalista que personal respectively mercadi or to fruit |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | i have the pleasure to present to you dr martin luther king there |
| parakeet.cpp f32 | i have the pleasure to present to you dr martin luther king there |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | and so my fellow americans ask not what your country can do for you ask what you can do for your country |
| parakeet.cpp f32 | and so my fellow americans ask not what your country can do for you ask what you can do for your country |
| Engine | Transcript |
|---|
| NeMo (PyTorch CPU) | hello this is a test of the voxtrol speech to text system |
| parakeet.cpp f32 | hello this is a test of the voxtrol speech to text system |
The FastConformer encoder uses global relative-position self-attention, which
is O(T²) in time and memory. On a long clip this explodes: a ~16.6-min file
subsamples to T≈12k encoder frames, and the score/mask tensors alone reach tens
of GB — enough to OOM a node. parakeet.cpp ports NeMo's rel_pos_local_attn
(Longformer-style banded attention, change_attention_model('rel_pos_local_attn',[W,W])):
each query attends only to keys within a ±W window, making attention O(T·W).
It auto-enables for long audio (encoder frames > 8192); PARAKEET_ATT_CONTEXT=W
forces a window (0 = full attention). The band is built with a chunk-matmul
construction (overlapping K/V chunks + one batched GEMM + a diagonal skew-view),
so the graph node count is independent of the window — the window goes to
NeMo's full [128,128] at no extra graph cost.
16.6-min clip (tdt-0.6b-v3, f32, NVIDIA GB10 DGX Spark, CPU / 16 threads):
| Attention | Window | Wall | RTFx | Peak RSS |
|---|
| full (global, O(T²)) | — | 148.3 s | 6.7× | 54.0 GB |
| banded | W=32 | 39.5 s | 25.2× | 8.9 GB |
| banded | W=128 (NeMo full) | 36.9 s | 27.0× | 9.4 GB |
Banded attention at NeMo's full W=128 is ~4× faster and ~5.7× less peak memory
than the global path, with a coherent transcript — and the chunk-matmul keeps the
wide window as cheap as the narrow one. Short clips (the LibriSpeech set above)
stay on the global path and are byte-identical to before; banding only engages
past the long-audio threshold.
parakeet.cpp reproduces NeMo with high fidelity: mean f32 agreement WER is 0.0155% — effectively byte-identical output, and WER vs ground truth tracks NeMo. The faithful path (f32) is the reference; quantization is opt-in for size.
parakeet.cpp is faster than NeMo on all 10 models (mean f32 speedup 1.40×, q8_0 1.56×; best: tdt_ctc-1.1b at 1.69×). The decisive decode-side win was caching the transducer prediction-net forward pass across non-emitting frames (the LSTM was ~97% of RNN-T decode time and mostly redundant); the encoder side uses a persistent ggml backend + gallocr, zero-copy weights, a single fused graph, and tinyBLAS (GGML_LLAMAFILE).
The ggml engine uses markedly less peak RAM than NeMo/PyTorch, and quantization lowers it further — q8_0 is near-lossless, while K-quants (q6_k→q4_k) keep shrinking the model on disk and in RAM at a small, monotonic accuracy cost (see the quantization plots).
- CPU-only; a CUDA/Metal ggml backend would change the picture again.
- NeMo is the unchanged reference (re-running its slow CPU pass reproduces the same numbers); only the ggml engine was re-measured for this refresh.
- Thread scaling (threads.json) shows 8 threads as the sweet spot on this 20-core host.