IMBE-ASR: Speech Recognition Directly from Vocoder Parameters

March 28, 2026 · View on GitHub

ASR straight from P25 IMBE codec parameters — skip audio reconstruction entirely and go from the digital bitstream to text. 290M Conformer-CTC, 3.35% WER with included language model on LibriSpeech-IMBE.

Models: trunk-reporter/imbe-asr collection on Hugging Face

ModelParamsGreedy WERBeam+LM WERUse caseHF Repo
Large290M6.5%3.35% (5-gram)Best accuracyimbe-asr-large-1024d
Base48.6M10.6%4.84% (3-gram)Edge / Pi 5imbe-asr-base-512d
Base P2548.6M37.1% → 19.2% beam19.2%Live P25 radioimbe-asr-base-512d-p25

All models ship with a tuned KenLM language model. ONNX (fp32, int8, uint8) + SafeTensors formats.

Quick Start

pip install onnxruntime numpy huggingface_hub pyctcdecode kenlm

# Live P25 radio (recommended for real radio traffic)
python3 scripts/transcribe_p25.py path/to/call.dvcf
python3 scripts/transcribe_p25.py --watch /path/to/dvcf/dir

# Best accuracy (LibriSpeech-IMBE, clean speech)
python3 scripts/transcribe_large.py path/to/call.dvcf

# Edge deployment (Raspberry Pi 5)
python3 scripts/transcribe_base.py path/to/call.dvcf

Models download automatically from Hugging Face on first run.

The Problem

P25 digital radio (law enforcement, fire, EMS) uses the IMBE vocoder -- a 4.4 kbps codec from the 90s. If you want to transcribe it, the standard approach is: decode IMBE -> reconstruct audio -> run Whisper or whatever.

But there is no public IMBE specification. The codec is proprietary (DVSI). libimbe -- the open-source implementation everyone uses -- is a reverse-engineered best guess at the original algorithm. The audio it produces is an approximation of an approximation. So the standard pipeline is:

codec params -> proprietary-spec-approximated waveform reconstruction -> mel spectrogram extraction -> ASR

Every step is lossy. The reconstruction itself is computationally expensive (trig, IFFT, overlap-add synthesis per frame), and at the end you've created a degraded audio signal so a speech model can re-extract spectral features -- features that were already there in the codec parameters.

We skip all of that.

Why Skip Reconstruction

The vocoder parameters already encode speech: fundamental frequency, spectral amplitudes, voicing decisions. That IS the speech, in a compact parametric representation. The model doesn't need a waveform -- it needs phonetic information, which the codec parameters describe directly.

The computational argument: IMBE waveform synthesis requires per-frame harmonic oscillator evaluation, spectral amplitude interpolation, voiced/unvoiced mixing, and overlap-add windowing. Then you still need a spectrogram. Our approach is a single matrix multiply (170-dim -> 1024-dim) per frame. Full inference on a 10-second call: 5.8ms on GPU (200x real-time). The vocoder reconstruction alone would take longer.

We also sidestep the fidelity question entirely. It doesn't matter how accurate libimbe's reconstruction is, because we never reconstruct. We go straight from the digital bitstream to text.

Input Representation

Each 20ms IMBE frame is expanded into a 170-dimensional vector:

[0]       f0 (fundamental frequency)
[1]       L (harmonic count, 0-56)
[2:58]    spectral amplitudes (56 slots, zero-padded)
[58:114]  voiced/unvoiced flags per harmonic
[114:170] binary validity mask (1=real harmonic, 0=padding)

The mask is critical -- without it the model can't distinguish silence from zero-padded harmonics. We tried raw 8-dim codewords (too compressed) and 20-dim BFCCs (lost information). This 170-dim representation won decisively.

Results

LibriSpeech-IMBE validation (speaker-split, 2775 utterances):

ModelGreedy WERBeam+LM WERLMParams
Large (290M)6.5%3.35%5-gram trie+q8 (α=0.7, β=2.0)290M
Base (48.6M)10.6%4.84%3-gram trie+q8 (α=0.7, β=2.0)48.6M

Real P25 radio (50 labeled samples, law enforcement/fire/EMS):

ModelGreedy WERBeam+LM WERLMNotes
Base P25 (48.6M)37.1%19.2%3-gram trie+q8 (α=0.5, β=1.0)P25 fine-tuned

For context: this is from 4.4 kbps vocoder parameters, not audio. The IMBE codec was designed in 1993 to be barely intelligible to human ears. Language models are included in each HF repo and tuned per model.

Full eval logs with REF/HYP examples are in results/.

Scaling progression:

ModelParamsGreedy WER
Proof of concept (d=256, 6L)9M40.6%
Base (d=512, 8L)48.6M18.9%
Large (d=1024, 12L)290M6.5%

Training curve (290M model, 30 epochs on ~1220h IMBE-encoded speech):

Epoch  2: WER=55.2%  val_loss=0.873
Epoch 10: WER=25.4%  val_loss=0.443
Epoch 20: WER=16.4%  val_loss=0.350
Epoch 30: WER=14.7%  val_loss=0.379

Training WER is on the full multi-source val set (LibriSpeech + TEDLIUM + GigaSpeech). The 6.5%/1.9% numbers are on the LibriSpeech speaker-split val, comparable to standard benchmarks. Full training curve in results/training_curve_1024d.txt.

W&B run: sarah-1024d-12l-ddp

Inference speed (GPU workstation):

Call DurationGPU (3090 Ti)CPU
1s5.8ms12ms
5s5.8ms25ms
10s5.8ms49ms
30s6.9ms241ms

200x real-time on CPU, 1700x on GPU.

Edge Deployment (Raspberry Pi 5, 4GB)

The 290M model runs on a $60 Raspberry Pi via ONNX Runtime + int8 quantization. PyTorch can't even load it (OOM at 3.3GB), but quantized to 298-312MB it fits comfortably.

RuntimeModelSize10s callRTFRAM
C (70KB binary)48.6M fp32195 MB660ms0.07x~300 MB
C (70KB binary)48.6M uint859 MB788ms0.08x~140 MB
C (70KB binary)290M uint8312 MB2.8s0.28x~1.3 GB
Python ONNX RT48.6M fp32195 MB649ms0.07x285 MB
Python ONNX RT290M int8298 MB3.5s0.35x535 MB
PyTorch48.6M fp32567 MB800ms0.08x995 MB
PyTorch290M fp323.3 GBOOM-->4 GB

All models faster than real-time. The C engine is a 70KB binary -- no Python needed. int8 quantization preserves perfect accuracy on the 290M model.

Full benchmark logs in results/.

Architecture

Conformer-CTC. Nothing exotic -- the interesting part is the input, not the model.

  • Input projection: Linear(170, 1024) + LayerNorm + Dropout
  • Encoder: 12 x ConformerBlock (FFN/2 -> MHSA -> ConvModule(k=31) -> FFN/2 -> LN)
  • CTC head: Linear(1024, 40) -> log_softmax
  • 290M parameters
  • Character-level tokenizer: blank + A-Z + 0-9 + space + apostrophe = 40 classes

Trained with CTC loss, cosine LR schedule, AdamW, bf16 mixed precision on 2x RTX 3090 Ti (~4 days).

Bayesian hyperparameter sweep confirmed depth matters more than width. Optimal: d=1024, 12 layers, ff_mult=4.

Training Data

~1220 hours of clean speech (LibriSpeech 960h, TEDLIUM 3 ~452h, GigaSpeech S ~250h), IMBE-encoded through libimbe to produce 170-dim frame parameters. The model learns to read IMBE parameters the way a conventional ASR model reads mel spectrograms.

Subtle detail: software-encoded IMBE has a 2-frame analysis delay (frame N describes audio at frame N-2). Real P25 radio doesn't have this. All software-encoded training data is trimmed accordingly.

For P25 domain adaptation: ~20 hours of real radio captures, pseudo-labeled with a Whisper large-v3 + Qwen3-ASR ensemble, filtered for hallucinations.

Reproducing

# Precompute 170-dim IMBE features from audio
python -m src.precompute --pairs-dir data/pairs --workers 12

# Train (multi-source, DDP)
torchrun --nproc_per_node=2 -m src.train \
    --mmap-dir data/packed \
    --d-model 1024 --n-layers 12 --n-heads 16 --d-ff 4096 \
    --epochs 30 --batch-size 4 --accum-steps 32 --lr 3e-4

# Evaluate (greedy)
python -m src.eval checkpoints/best.pth \
    --pairs-dir data/pairs --batch-size 16

# Evaluate (beam search + LM)
python -m src.eval checkpoints/best.pth \
    --beam --lm-path data/lm/5gram.bin --unigrams data/lm/unigrams.txt

# Inference on a P25 .dvcf file
python -m src.inference --checkpoint checkpoints/best.pth \
    --dvcf-file path/to/call.dvcf

# Watch directory for live transcription
python -m src.inference --checkpoint checkpoints/best.pth \
    --watch ~/trunk-recorder/audio/

# P25 fine-tuning (DDP)
torchrun --nproc_per_node=2 scripts/finetune_p25.py \
    --checkpoint checkpoints/best.pth \
    --p25-dir data/p25_labeled --base-mmap data/packed \
    --epochs 15 --lr 3e-5 --amp

Project Structure

src/
  model.py            Conformer-CTC architecture (scalable: 9M to 290M)
  tokenizer.py        Character-level CTC tokenizer (40 classes)
  dataset.py          LibriSpeech IMBE dataset + speaker splits
  dataset_unified.py  Multi-source dataset + memory-mapped format
  dataset_p25.py      Real P25 dataset (talkgroup splits)
  train.py            Training loop (CTC, cosine LR, DDP multi-GPU)
  eval.py             WER/CER evaluation (greedy + beam)
  decode.py           Beam search with KenLM
  inference.py        TAP file / watch mode / streaming inference
  symbolstream_client.py  Live transcription via symbolstream plugin (TCP)
  live.py             Legacy Unix socket transcription
  precompute.py       Audio -> 170-dim IMBE features via libimbe

scripts/
  finetune_p25.py     P25 fine-tuning with base data mixing (DDP)
  pseudo_label.py     Multi-ASR pseudo-labeling for P25
  prepare_*.py        Dataset preparation scripts
  build_lm.py         KenLM language model training

results/              Eval logs and training curves
configs/              Training configurations

Getting IMBE Symbols from P25 Radio

Important: Standard trunk-recorder only outputs reconstructed audio. To use IMBE-ASR, you need the raw IMBE codec symbols (codewords) before audio reconstruction.

Our fork of trunk-recorder adds a voice_codec_data() callback that exposes the raw IMBE frame vectors. The symbolstream plugin uses this callback to stream raw codec symbols over TCP/UDP to a remote server -- the same way simplestream streams audio, but with the pre-vocoder codec parameters instead. This is what the model consumes: the 8 codeword parameters per 20ms frame, decoded into 170-dim features via libimbe.

Without this fork and plugin (or another source of raw IMBE symbols), the models cannot be used on live radio. The whole point is to skip audio reconstruction -- if you only have audio, use a conventional ASR model like Whisper or our Qwen3-ASR P25 fine-tune.

API Server

Run the IMBE-ASR transcription server (OpenAI-compatible endpoint):

# With SafeTensors checkpoint (recommended)
IMBE_ASR_CHECKPOINT=model.safetensors uvicorn server.app:app --port 8000

# With ONNX checkpoint (for edge/CPU deployment)
IMBE_ASR_CHECKPOINT=model_int8.onnx uvicorn server.app:app --port 8000

# Docker
docker build -f server/Dockerfile -t imbe-asr-server .
docker run -p 8000:8000 \
    -v /path/to/models:/models \
    -e IMBE_ASR_CHECKPOINT=/models/model.safetensors \
    imbe-asr-server

Endpoints:

  • POST /v1/audio/transcriptions — upload a .dvcf file, get text back
  • GET /health — model status, format, device info

Supports all checkpoint formats: SafeTensors (.safetensors), ONNX (.onnx), and PyTorch (.pth).

Integrates with tr-engine as an STT provider — set STT_PROVIDER=imbe and IMBE_ASR_URL=http://host:8000.

Live transcription with symbolstream

Start the symbolstream client (it listens for the plugin to connect):

python -m src.symbolstream_client \
    --checkpoint model.safetensors \
    --port 9090 --stream

Then configure the symbolstream plugin in trunk-recorder's config.json to point at this machine:

{
    "name": "symbolstream",
    "library": "libsymbolstream_plugin",
    "streams": [{
        "address": "<this machine's IP>",
        "port": 9090,
        "TGID": 0,
        "useTCP": true,
        "sendJSON": true
    }]
}

Dependencies

  • Python 3.10+, PyTorch 2.0+
  • libimbe.so -- IMBE vocoder C library (for feature extraction from codewords)
  • trunk-recorder fork (for live P25 symbol capture)
  • safetensors (SafeTensors checkpoint loading)
  • onnxruntime (ONNX checkpoint loading, optional)
  • pyctcdecode + kenlm (beam search decoding)
  • wandb (experiment tracking)
  • fastapi + uvicorn (API server)

Beyond IMBE

The approach is codec-agnostic. IMBE and AMBE/AMBE+2 are the same family (both DVSI) -- same parametric structure: f0, spectral amplitudes, voiced/unvoiced decisions. Swapping libimbe for libambe in feature extraction is the only change needed. Not yet tested, but if it transfers, the coverage is significant:

CodecBitrateSystems
IMBE4.4 kbpsP25 Phase 1
AMBE+22.4-9.6 kbpsP25 Phase 2, DMR, dPMR, NXDN, D-STAR, Fusion
Codec20.7-3.2 kbpsFreeDV, open-source HF radio
MELPe1.2-2.4 kbpsNATO STANAG 4591, military

DMR alone covers most commercial/industrial two-way radio worldwide. D-STAR is amateur radio. NXDN is Kenwood/Icom. If the principle holds -- that speech codecs are already doing feature extraction and we just need to learn to read their output -- this generalizes well beyond P25.

Roadmap

See the Trunk Reporter Roadmap for the cross-repo project tracker with priorities and phases.

What's Next

  • P25 fine-tuning the 290M model (in progress)
  • AMBE+2 support (DMR, P25 Phase 2, D-STAR)
  • tr-engine integration via MQTT dvcf transport (in progress)