Speculative Decoding

Speculative decoding accelerates DLLM inference by drafting candidate tokens and verifying them with the target model.

Supported Methods

Method	Paper	Config
LoPA	Paper	generation=lopa
FreeDAVE	Paper	generation=freedave

Quick Start

Run all speculative-decoding examples on LLaDA and GSM8K:

bash scripts/run_speculative_decoding.sh

Method Examples

The examples below use model=llada-inst and dataset.name=gsm8k. To run the same commands with Dream, use model=dream-inst and set DREAM_INST_PATH; to switch benchmarks, replace dataset.name=gsm8k with any task available through lm-eval or the local tasks/ directory. Example tasks include humaneval_instruct, math-500, mbpp_instruct, ifeval, gpqa_main_generative_n_shot, and longbench.

The current LoPA and FreeDAVE implementations are not wired together with cache methods yet, although such combinations are possible in principle.

LoPA

accelerate launch \
    --num_machines 1 \
    --num_processes 1 \
    eval.py \
    dataset.name=gsm8k \
    batch_size=1 \
    seed=1234 \
    attn_implementation=sdpa \
    flash_attention=true \
    generation=lopa \
    generation.k=5 \
    generation.threshold=0.9 \
    generation.block_length=32 \
    generation.gen_length=256 \
    generation.steps=256 \
    model=llada-inst \
    hydra.run.dir=./outputs/examples/speculative_decoding/llada-inst-lopa-gsm8k

Note: Defaults are generation.k=5 and generation.threshold=0.9; k sets the proposal width, and threshold controls candidate acceptance.

FreeDAVE

accelerate launch \
    --num_machines 1 \
    --num_processes 1 \
    eval.py \
    dataset.name=gsm8k \
    batch_size=1 \
    seed=1234 \
    attn_implementation=sdpa \
    flash_attention=true \
    generation=freedave \
    generation.draft_steps=4 \
    generation.block_length=32 \
    generation.gen_length=256 \
    generation.steps=256 \
    model=llada-inst \
    hydra.run.dir=./outputs/examples/speculative_decoding/llada-inst-freedave-gsm8k

Note: Defaults are generation.draft_steps=4 and generation.confidence_threshold=null; draft_steps controls how many stale draft denoising steps are proposed before verification. With confidence_threshold=null, FreeDAVE uses schedule-driven top-k drafting from the denoising schedule. Setting generation.confidence_threshold enables threshold-first drafting: tokens with confidence greater than or equal to the threshold are drafted first, with scheduled top-k selection as a fallback when no token passes the threshold.