README.md

November 10, 2025 · View on GitHub


Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking

Pengxiang Li1†, Shilin Yan2†♠, Joey Tsai3, Renrui Zhang4, Ruichuan An5,
Ziyu Guo4, Xiaowei Gao6‡

1DLUT  2Alibaba  3Tsinghua  4CUHK  5PKU  6ICL

†Equal contribution   ♠Project leader   ‡Corresponding author

A-CFG is an adaptive version of Classifier-Free Guidance for diffusion-based language models. Instead of a static unconditional input, A-CFG dynamically re-masks low-confidence tokens at every denoising step, focusing guidance precisely where the model is uncertain.

✨ Key Features

  • Plug-and-play guidance module for any masked diffusion language model (e.g. LLaDA, Dream).
  • Token-level confidence heuristics with a single hyper-parameter ρ (remask ratio).

🚀 Quick Start

This project builds on LLaDA. See their README for more details on the base model setup.

LLaDA Inference

The LLaDA-8B-Base and LLaDA-8B-Instruct are uploaded in Huggingface. Please first install transformers==4.38.2 and employ the transformers to load.

from transformers import AutoModel, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained('GSAI-ML/LLaDA-8B-Base', trust_remote_code=True)
model = AutoModel.from_pretrained('GSAI-ML/LLaDA-8B-Base', trust_remote_code=True, torch_dtype=torch.bfloat16)

📄 Cite

@article{li2025adaptive,
  title={Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking},
  author={Li, Pengxiang and Yan, Shilin and Tsai, Joey and Zhang, Renrui and An, Ruichuan and Guo, Ziyu and Gao, Xiaowei},
  journal={arXiv preprint arXiv:2505.20199},
  year={2025}
}