README.md

This repository is for our paper:

Discrete Diffusion in Large Language and Multimodal Models: A Survey
Runpeng Yu*¹, Qi Li*¹, Xinchao Wang^†1
1National University of Singapore, Singapore
^*Equal contribution, random order.
^†Corresponding author: xinchao@nus.edu.sg

Figure 1. Difference between autoregressive models and discrete diffusion models.

Figure 2. A timeline of existing dLLMs and dMLLMs in recent years.

Citation

@article{yu2025discrete,
  title={Discrete Diffusion in Large Language and Multimodal Models: A Survey},
  author={Yu, Runpeng and Li, Qi and Wang, Xinchao},
  journal={arXiv preprint arXiv:2506.13759},
  year={2025}
}

---

🌟 If you find this resource helpful, please consider to star this repository and cite our work!

🙋 Please let us know if you find out a mistake or have any suggestions!

To add new papers or models

If you want to add your paper or update details like conference info or code URLs, please raise an PR. You can generate the necessary markdown for each paper by filling out generate_item.py and running python generate_item.py. We greatly appreciate your contributions. Alternatively, you can emal me (liqi@u.nus.edu) the links to your paper and code, and I will add your paper to the list as soon as possible.

---

Quick Links

• Mathematics
  • Discrete Diffusion Model and Transition Matrix
  • Simplified Masked Diffusion Model
  • Continuous Time Discrete Denoising Models
  • Concrete Score
  • Reparameterized Discrete Diffusion Model
• Modeling Language Diffusion
• Representative Model
  • Discrete Diffusion Models around 1B
  • Large Diffusion Language Models
  • Large Diffusion Multimodal Models
  • Large Unified Models
• Training Techniques
  • Initialization Technique
  • Complementary Masking Technique
  • Masking Scheduling Technique
  • Reweighting Technique
  • Distillation
  • Training-Testing Input Discrepancy
• Inference Techniques
  • Unmasking Techniques
  • Remasking Techniques
  • Prefilling and Caching Technique
  • Guidance Technique
  • Sampling Technique
  • Context Length Extension
  • Sparse Computation
  • Response Length Control
• Quantization
• Privacy and Safety
• Applications
  • Text Generation and Style Control
  • Text Editing and Summarization
  • Sentiment Analysis and Data Augmentation
  • Knowledge and Reasoning
  • Vision and Multimodal
  • Biological and Drug Discovery
• Toolkits

Mathematics

Discrete Diffusion Model and Transition Matrix

🔝 Back to Top

Title & Authors	Introduction	Links
Deep unsupervised learning using nonequilibrium thermodynamics Sohl-Dickstein, Jascha, Eric Weiss, Niru Maheswaranathan, Surya Ganguli		Paper
Structured denoising diffusion models in discrete state-spaces Austin, Jacob, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, Rianne Van Den Berg		Paper
Argmax flows and multinomial diffusion: Learning categorical distributions Hoogeboom, Emiel, Didrik Nielsen, Priyank Jaini, Patrick Forré, Max Welling		Paper
Generalized interpolating discrete diffusion Dimitri von Rütte, Janis Fluri, Yuhui Ding, Antonio Orvieto, Bernhard Schölkopf, Thomas Hofmann		Paper

Simplified Masked Diffusion Model

🔝 Back to Top

Title & Authors	Introduction	Links
Simple and Effective Masked Diffusion Language Models Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, Volodymyr Kuleshov		Paper
Simplified and Generalized Masked Diffusion for Discrete Data Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, Michalis K. Titsias		Paper

Continuous Time Discrete Denoising Models

🔝 Back to Top

Title & Authors	Introduction	Links
A Continuous Time Framework for Discrete Denoising Models Andrew Campbell, Joe Benton, Valentin De Bortoli, Tom Rainforth, George Deligiannidis, Arnaud Doucet		Paper

Concrete Score

🔝 Back to Top

Title & Authors	Introduction	Links
Concrete Score Matching: Generalized Score Matching for Discrete Data Chenlin Meng, Kristy Choi, Jiaming Song, Stefano Ermon		Paper
Score-based Continuous-time Discrete Diffusion Models Haoran Sun, Lijun Yu, Bo Dai, Dale Schuurmans, Hanjun Dai		Paper
Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Data Jingyang Ou, Shen Nie, Kaiwen Xue, Fengqi Zhu, Jiacheng Sun, Zhenguo Li, Chongxuan Li		Paper
Target concrete score matching: A holistic framework for discrete diffusion Ruixiang Zhang, Shuangfei Zhai, Yizhe Zhang, James Thornton, Zijing Ou, Joshua Susskind, Navdeep Jaitly		Paper
Simple and Effective Masked Diffusion Language Models Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, Volodymyr Kuleshov		Paper
Efficient perplexity bound and ratio matching in discrete diffusion language models Etrit Haxholli, Yeti Z. Gurbuz, Oğul Can, Eli Waxman		Paper

Reparameterized Discrete Diffusion Model

🔝 Back to Top

Title & Authors	Introduction	Links
A Reparameterized Discrete Diffusion Model for Text Generation Lin Zheng, Jianbo Yuan, Lei Yu, Lingpeng Kong		Paper

Modeling Language Diffusion

🔝 Back to Top

Title & Authors	Introduction	Links
Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models Marianne Arriola, Aaron Gokaslan, Justin T. Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, Volodymyr Kuleshov		Paper
Any-Order Flexible Length Masked Diffusion Jaeyeon Kim, Lee Cheuk-Kit, Carles Domingo-Enrich, Yilun Du, Sham Kakade, Timothy Ngotiaoco, Sitan Chen, Michael Albergo		Paper
Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Masking Chen-Hao Chao, Wei-Fang Sun, Hanwen Liang, Chun-Yi Lee, Rahul G. Krishnan		Paper
Flexible-length Text Infilling for Discrete Diffusion Models Andrew Zhang, Anushka Sivakumar, Chiawei Tang, Chris Thomas		Paper

Representative Model

Discrete Diffusion Models around 1B

🔝 Back to Top

Title & Authors	Introduction	Links
Structured Denoising Diffusion Models in Discrete State-Spaces Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, Rianne van den Berg		Paper
DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models Zhengfu He, Tianxiang Sun, Kuanning Wang, Xuanjing Huang, Xipeng Qiu		Paper
A Reparameterized Discrete Diffusion Model for Text Generation Lin Zheng, Jianbo Yuan, Lei Yu, Lingpeng Kong		Paper
A Cheaper and Better Diffusion Language Model with Soft-Masked Noise Jiaao Chen, Aston Zhang, Mu Li, Alex Smola, Diyi Yang		Paper
Diffusion-NAT: Self-Prompting Discrete Diffusion for Non-Autoregressive Text Generation Kun Zhou, Yifan Li, Wayne Xin Zhao, Ji-Rong Wen		Paper
TESS: Text-to-Text Self-Conditioned Simplex Diffusion Rabeeh Karimi Mahabadi, Hamish Ivison, Jaesung Tae, James Henderson, Iz Beltagy, Matthew E. Peters, Arman Cohan		Paper
Likelihood-Based Diffusion Language Models Ishaan Gulrajani, Tatsunori B. Hashimoto		Paper
Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution Aaron Lou, Chenlin Meng, Stefano Ermon		Paper
Simple and Effective Masked Diffusion Language Models Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T Chiu, Alexander Rush, Volodymyr Kuleshov		Paper
Simplified and Generalized Masked Diffusion for Discrete Data Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, Michalis K. Titsias		Paper
Unified Multimodal Discrete Diffusion Alexander Swerdlow, Mihir Prabhudesai, Siddharth Gandhi, Deepak Pathak, Katerina Fragkiadaki		Paper

Large Diffusion Language Models

🔝 Back to Top

Title & Authors	Introduction	Links
Large Language Diffusion Models Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, Chongxuan Li		Paper
Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuning Jiasheng Ye, Zaixiang Zheng, Yu Bao, Lihua Qian, Quanquan Gu		Paper
Scaling Diffusion Language Models via Adaptation from Autoregressive Models Shansan Gong, Shivam Agarwal, Yizhe Zhang, Jiacheng Ye, Lin Zheng, Mukai Li, Chenxin An, Peilin Zhao, Wei Bi, Jiawei Han, Hao Peng, Lingpeng Kong		Paper
Dream 7b Jiacheng Ye*, Zhihui Xie*, Lin Zheng*, Jiahui Gao*, Zirui Wu, Xin Jiang, Zhenguo Li, Lingpeng Kong		Paper
LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, Chongxuan Li		Paper
TESS 2: A Large-Scale Generalist Diffusion Language Model Jaesung Tae, Hamish Ivison, Sachin Kumar, Arman Cohan		Paper
DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvas Zirui Wu*, Lin Zheng*, Zhihui Xie, Jiacheng Ye, Jiahui Gao, Yansong Feng, Zhenguo Li, Victoria W., Guorui Zhou , Lingpeng Kong		Paper
Dream-Coder 7B: An Open Diffusion Language Model for Code Zhihui Xie, Jiacheng Ye, Lin Zheng, Jiahui Gao, Jingwei Dong, Zirui Wu, Xueliang Zhao, Shansan Gong, Xin Jiang, Zhenguo Li, Lingpeng Kong		Paper
DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation Shansan Gong, Ruixiang Zhang, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong, Yizhe Zhang		Paper
Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference Yuxuan Song, Zheng Zhang, Cheng Luo, Pengyang Gao, Fan Xia, Hao Luo, Zheng Li, Yuehang Yang, Hongli Yu, Xingwei Qu, Yuwei Fu, Jing Su, Ge Zhang, Wenhao Huang, Mingxuan Wang, Lin Yan, Xiaoying Jia, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Yonghui Wu, Hao Zhou		Paper

Large Diffusion Multimodal Models

🔝 Back to Top

Title & Authors	Introduction	Links
Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding Runpeng Yu, Xinyin Ma, Xinchao Wang		Paper
LaViDa: A Large Diffusion Language Model for Multimodal Understanding Shufan Li, Konstantinos Kallidromitis, Hritik Bansal, Akash Gokul, Yusuke Kato, Kazuki Kozuka, Jason Kuen, Zhe Lin, Kai-Wei Chang, Aditya Grover		Paper
LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning Zebin You, Shen Nie, Xiaolu Zhang, Jun Hu, Jun Zhou, Zhiwu Lu, Ji-Rong Wen, Chongxuan Li		Paper

Large Unified Models

🔝 Back to Top

Title & Authors	Introduction	Links
MMaDA: Multimodal Large Diffusion Language Models Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, Mengdi Wang		Paper
FUDOKI: Discrete Flow-based Unified Understanding and Generation via Kinetic-Optimal Velocities Jin Wang, Yao Lai, Aoxue Li, Shifeng Zhang, Jiacheng Sun, Ning Kang, Chengyue Wu, Zhenguo Li, Ping Luo		Paper
Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Model Qingyu Shi, Jinbin Bai, Zhuoran Zhao, Wenhao Chai, Kaidong Yu, Jianzong Wu, Shuangyong Song, Yunhai Tong, Xiangtai Li, Xuelong Li, Shuicheng Yan		Paper

Training Techniques

Initialization Technique

🔝 Back to Top

Title & Authors	Introduction	Links
DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models Zhengfu He, Tianxiang Sun, Kuanning Wang, Xuanjing Huang, Xipeng Qiu		Paper
Scaling Diffusion Language Models via Adaptation from Autoregressive Models Shansan Gong, Shivam Agarwal, Yizhe Zhang, Jiacheng Ye, Lin Zheng, Mukai Li, Chenxin An, Peilin Zhao, Wei Bi, Jiawei Han, Hao Peng, Lingpeng Kong		Paper
Dream 7b Jiacheng Ye*, Zhihui Xie*, Lin Zheng*, Jiahui Gao*, Zirui Wu, Xin Jiang, Zhenguo Li, Lingpeng Kong		Paper
Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding Runpeng Yu, Xinyin Ma, Xinchao Wang		Paper

Complementary Masking Technique

🔝 Back to Top

Title & Authors	Introduction	Links
LaViDa: A Large Diffusion Language Model for Multimodal Understanding Shufan Li, Konstantinos Kallidromitis, Hritik Bansal, Akash Gokul, Yusuke Kato, Kazuki Kozuka, Jason Kuen, Zhe Lin, Kai-Wei Chang, Aditya Grover		Paper

Masking Scheduling Technique

🔝 Back to Top

Title & Authors	Introduction	Links
Structured denoising diffusion models in discrete state-spaces Austin, Jacob, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, Rianne Van Den Berg		Paper
Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution Aaron Lou, Chenlin Meng, Stefano Ermon		Paper
MaskGIT: Masked Generative Image Transformer Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, William T. Freeman		Paper
DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models Zhengfu He, Tianxiang Sun, Kuanning Wang, Xuanjing Huang, Xipeng Qiu		Paper

Reweighting Technique

🔝 Back to Top

Title & Authors	Introduction	Links
Beyond autoregression: Discrete diffusion for complex reasoning and planning Jiacheng Ye, Jiahui Gao, Shansan Gong, Lin Zheng, Xin Jiang, Zhenguo Li, Lingpeng Kong		Paper

Distillation

🔝 Back to Top

Title & Authors	Introduction	Links
Distillation of Discrete Diffusion through Dimensional Correlations Satoshi Hayakawa, Yuhta Takida, Masaaki Imaizumi, Hiromi Wakaki, Yuki Mitsufuji		Paper
dParallel: Learnable Parallel Decoding for dLLMs Zigeng Chen, Gongfan Fang, Xinyin Ma, Ruonan Yu, Xinchao Wang		Paper

Reinforcement Learning

🔝 Back to Top

Title & Authors	Introduction	Links
MMaDA: Multimodal Large Diffusion Language Models Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, Mengdi Wang		Paper
LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, Chongxuan Li		Paper
Discrete Diffusion Trajectory Alignment via Stepwise Decomposition Jiaqi Han, Austin Wang, Minkai Xu, Wenda Chu, Meihua Dang, Yisong Yue, Stefano Ermon		Paper
wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models Xiaohang Tang, Rares Dolga, Sangwoong Yoon, Ilija Bogunovic		Paper
Reinforcing the Diffusion Chain of Lateral Thought with Diffusion Language Models Zemin Huang, Zhiyang Chen, Zijun Wang, Tiancheng Li, Guo-Jun Qi		Paper

Training-Testing Input Discrepancy

🔝 Back to Top

Title & Authors	Introduction	Links
Addressing the Training-Inference Discrepancy in Discrete Diffusion for Text Generation Masaki Asada, Makoto Miwa		Paper

Inference Techniques

Unmasking Techniques

🔝 Back to Top

Title & Authors	Introduction	Links
Dream 7b Jiacheng Ye*, Zhihui Xie*, Lin Zheng*, Jiahui Gao*, Zirui Wu, Xin Jiang, Zhenguo Li, Lingpeng Kong		Paper
Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions Jaeyeon Kim, Kulin Shah, Vasilis Kontonis, Sham Kakade, Sitan Chen		Paper
Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding Runpeng Yu, Xinyin Ma, Xinchao Wang		Paper
LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuning Zebin You, Shen Nie, Xiaolu Zhang, Jun Hu, Jun Zhou, Zhiwu Lu, Ji-Rong Wen, Chongxuan Li		Paper
Discrete Flow Matching Itai Gat, Tal Remez, Neta Shaul, Felix Kreuk, Ricky T. Q. Chen, Gabriel Synnaeve, Yossi Adi, Yaron Lipman		Paper
Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking Heli Ben-Hamu, Itai Gat, Daniel Severo, Niklas Nolte, Brian Karrer		Paper
Pc-sampler: Position-aware calibration of decoding bias in masked diffusion models Pengcheng Huang, Shuhao Liu, Zhenghao Liu, Yukun Yan, Shuo Wang, Zulong Chen, Tong Xiao		Paper
Plan for Speed--Dilated Scheduling for Masked Diffusion Language Models Omer Luxembourg, Haim Permuter, Eliya Nachmani		Paper

Remasking Techniques

🔝 Back to Top

Title & Authors	Introduction	Links
Remasking Discrete Diffusion Models with Inference-Time Scaling Guanghan Wang, Yair Schiff, Subham Sekhar Sahoo, Volodymyr Kuleshov		Paper
Discrete Flow Matching Itai Gat, Tal Remez, Neta Shaul, Felix Kreuk, Ricky T. Q. Chen, Gabriel Synnaeve, Yossi Adi, Yaron Lipman		Paper
Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMs Feng Hong, Geng Yu, Yushi Ye, Haicheng Huang, Huangjie Zheng, Ya Zhang, Yanfeng Wang, Jiangchao Yao		Paper

Prefilling and Caching Technique

🔝 Back to Top

Title & Authors	Introduction	Links
Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding Runpeng Yu, Xinyin Ma, Xinchao Wang		Paper
LaViDa: A Large Diffusion Language Model for Multimodal Understanding Shufan Li, Konstantinos Kallidromitis, Hritik Bansal, Akash Gokul, Yusuke Kato, Kazuki Kozuka, Jason Kuen, Zhe Lin, Kai-Wei Chang, Aditya Grover		Paper
dKV-Cache: The Cache for Diffusion Language Models Xinyin Ma, Runpeng Yu, Gongfan Fang, Xinchao Wang		Paper
dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching Zhiyuan Liu, Yicun Yang, Yaojie Zhang, Junjie Chen, Chang Zou, Qingyuan Wei, Shaobo Wang, Linfeng Zhang		Paper
Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, Enze Xie		Paper

Guidance Technique

🔝 Back to Top

Title & Authors	Introduction	Links
Guided Transfer Learning for Discrete Diffusion Models Julian Kleutgens, Claudio Battiloro, Lingkai Kong, Benjamin Grewe, Francesca Dominici, Mauricio Tec		Github Paper
Scaling up Masked Diffusion Models on Text Shen Nie, Fengqi Zhu, Chao Du, Tianyu Pang, Qian Liu, Guangtao Zeng, Min Lin, Chongxuan Li		Paper
CtrlDiff: Boosting Large Diffusion Language Models with Dynamic Block Prediction and Controllable Generation Chihan Huang, Hao Tang		Paper
TESS 2: A Large-Scale Generalist Diffusion Language Model Jaesung Tae, Hamish Ivison, Sachin Kumar, Arman Cohan		Paper
Energy-based diffusion language models for text generation Minkai Xu, Tomas Geffner, Karsten Kreis, Weili Nie, Yilun Xu, Jure Leskovec, Stefano Ermon, Arash Vahdat		Paper

Sampling Technique

|Diffusion Language Models Know the Answer Before Decoding
Pengxiang Li, Yefan Zhou, Dilxat Muhtar, Lu Yin, Shilin Yan, Li Shen, Yi Liang, Soroush Vosoughi, Shiwei Liu | |Paper|[//]: #10/01 |Thinking Inside the Mask: In-Place Prompting in Diffusion LLMs
Xiangqi Jin, Yuxuan Wang, Yifeng Gao, Zichen Wen, Biqing Qi, Dongrui Liu, Linfeng Zhang | |Paper|[//]: #10/01 |Inference-Time Scaling of Diffusion Language Models with Particle Gibbs Sampling
Meihua Dang, Jiaqi Han, Minkai Xu, Kai Xu, Akash Srivastava, Stefano Ermon | |Paper|[//]: #10/01 |Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models
Wen Wang, Bozhen Fang, Chenchen Jing, Yongliang Shen, Yangyi Shen, Qiuyu Wang, Hao Ouyang, Hao Chen, Chunhua Shen | |Paper|[//]: #10/01

Context Length Extension

|LongLLaDA: Unlocking Long Context Capabilities in Diffusion LLMs
Xiaoran Liu, Zhigeng Liu, Zengfeng Huang, Qipeng Guo, Ziwei He, Xipeng Qiu | |Paper|[//]: #10/01

Sparse Computation

|Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction
Yuerong Song, Xiaoran Liu, Ruixiao Li, Zhigeng Liu, Zengfeng Huang, Qipeng Guo, Ziwei He, Xipeng Qiu | |Paper|[//]: #10/01 |DPad: Efficient Diffusion Language Models with Suffix Dropout
Xinhua Chen, Sitao Huang, Cong Guo, Chiyue Wei, Yintao He, Jianyi Zhang, Hai "Helen" Li, Yiran Chen | |Paper|[//]: #10/01 |Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing
Xu Wang, Chenkai Xu, Yijie Jin, Jiachun Jin, Hao Zhang, Zhijie Deng | |Paper|[//]: #10/01

Response Length Control

|Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Models
Jinsong Li, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Jiaqi Wang, Dahua Lin | |Paper|[//]: #10/01

Quantization

🔝 Back to Top

Title & Authors	Introduction	Links
Quantization Meets dLLMs: A Systematic Study of Post-training Quantization for Diffusion LLMs Haokun Lin, Haobo Xu, Yichen Wu, Ziyu Guo, Renrui Zhang, Zhichao Lu, Ying Wei, Qingfu Zhang, Zhenan Sun		Paper
DLLMQuant: Quantizing Diffusion-based Large Language Models Chen Xu, Dawei Yang		Paper

Privacy and Safety

🔝 Back to Top

Title & Authors	Introduction	Links
Every Step Counts: Decoding Trajectories as Authorship Fingerprints of dLLMs Qi Li, Runpeng Yu, Haiquan Lu, Xinchao Wang		Paper
The Devil behind the mask: An emergent safety vulnerability of Diffusion LLMs Zichen Wen, Jiashu Qu, Dongrui Liu, Zhiyuan Liu, Ruixi Wu, Yicun Yang, Xiangqi Jin, Haoyun Xu, Xuyang Liu, Weijia Li, Chaochao Lu, Jing Shao, Conghui He, Linfeng Zhang		Paper
Jailbreaking Large Language Diffusion Models: Revealing Hidden Safety Flaws in Diffusion-Based Text Generation Yuanhe Zhang, Fangzhou Xie, Zhenhong Zhou, Zherui Li, Hao Chen, Kun Wang, Yufei Guo		Paper
Where to Start Alignment? Diffusion Large Language Model May Demand a Distinct Position Zhixin Xie, Xurui Song, Jun Luo		Paper

Applications

Text Generation and Style Control

🔝 Back to Top

Title & Authors	Introduction	Links
Fine-grained Text Style Transfer with Diffusion-Based Language Models Yiwei Lyu, Tiange Luo, Jiacheng Shi, Todd Hollon, Honglak Lee		Paper
Diffusion vs. Autoregressive Language Models: A Text Embedding Perspective Siyue Zhang, Yilun Zhao, Liyuan Geng, Arman Cohan, Anh Tuan Luu, Chen Zhao		Paper
Segment-Level Diffusion: A Framework for Controllable Long-Form Generation with Diffusion Language Models Xiaochen Zhu, Georgi Karadzhov, Chenxi Whitehouse, Andreas Vlachos		Paper
DiffusPoll: Conditional Text Diffusion Model for Poll Generation Le Cheng, Shuangyin Li		Paper
PoetryDiffusion: Towards Joint Semantic and Metrical Manipulation in Poetry Generation Zhiyuan Hu, Chumin Liu, Yue Feng, Anh Tuan Luu, Bryan Hooi		Paper

Text Editing and Summarization

🔝 Back to Top

Title & Authors	Introduction	Links
EdiText: Controllable Coarse-to-Fine Text Editing with Diffusion Language Models Che Hyun Lee, Heeseung Kim, Jiheum Yeom, Sungroh Yoon		Paper
Discrete Diffusion Language Model for Efficient Text Summarization Do Huu Dat, Do Duc Anh, Anh Tuan Luu, Wray Buntine		Paper
DiffETM: Diffusion Process Enhanced Embedded Topic Model Wei Shao, Mingyang Liu, Linqi Song		Paper
TermDiffuSum: A Term-guided Diffusion Model for Extractive Summarization of Legal Documents Xiangyun Dong, Wei Li, Yuquan Le, Zhangyue Jiang, Junxi Zhong, Zhong Wang		Paper

Sentiment Analysis and Data Augmentation

🔝 Back to Top

Title & Authors	Introduction	Links
CDAˆ2: Counterfactual Diffusion Augmentation for Cross-Domain Adaptation in Low-Resource Sentiment Analysis Dancheng Xin, Kaiqi Zhao, Jingyun Sun, Yang Li		Paper
An Effective Deployment of Diffusion LM for Data Augmentation in Low-Resource Sentiment Classification Zhuowei Chen, Lianxi Wang, Yuben Wu, Xinfeng Liao, Yujia Tian, Junyang Zhong		Paper
Pinpointing Diffusion Grid Noise to Enhance Aspect Sentiment Quad Prediction Linan Zhu, Xiangfan Chen, Xiaolei Guo, Chenwei Zhang, Zhechao Zhu, Zehai Zhou, Xiangjie Kong		Paper

Knowledge and Reasoning

🔝 Back to Top

Title & Authors	Introduction	Links
Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models Jiacheng Ye, Shansan Gong, Liheng Chen, Lin Zheng, Jiahui Gao, Han Shi, Chuan Wu, Xin Jiang, Zhenguo Li, Wei Bi, Lingpeng Kong		Paper
DiffuCOMET: Contextual Commonsense Knowledge Diffusion Silin Gao, Mete Ismayilzada, Mengjie Zhao, Hiromi Wakaki, Yuki Mitsufuji, Antoine Bosselut		Paper
DPCL-Diff: The Temporal Knowledge Graph Reasoning based on Graph Node Diffusion Model with Dual-Domain Periodic Contrastive Learning Yukun Cao, Lisheng Wang, Luobing Huang		Paper
d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning Siyan Zhao, Devaansh Gupta, Qinqing Zheng, Aditya Grover		Paper
Beyond autoregression: Discrete diffusion for complex reasoning and planning Jiacheng Ye, Jiahui Gao, Shansan Gong, Lin Zheng, Xin Jiang, Zhenguo Li, Lingpeng Kong		Paper
Neurosymbolic diffusion models Emile van Krieken, Pasquale Minervini, Edoardo Ponti, Antonio Vergari		Paper

Vision and Multimodal

🔝 Back to Top

Title & Authors	Introduction	Links
DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Driving Anqing Jiang, Yu Gao, Zhigang Sun, Yiru Wang, Jijun Wang, Jinghao Chai, Qian Cao, Yuweng Heng, Hao Jiang, Yunda Dong, Zongzheng Zhang, Xianda Guo, Hao Sun, Hao Zhao		Paper
Underwater Diffusion Attention Network with Contrastive Language-Image Joint Learning for Underwater Image Enhancement Afrah Shaahid, Muzammil Behzad		Paper
Underwater Diffusion Attention Network with Contrastive Language-Image Joint Learning for Underwater Image Enhancement Haoran He, Chenjia Bai, Ling Pan, Weinan Zhang, Bin Zhao, Xuelong Li		Paper
M2D2M: Multi-Motion Generation from Text with Discrete Diffusion Models Seunggeun Chi, Hyung-gun Chi, Hengbo Ma, Nakul Agarwal, Faizan Siddiqui, Karthik Ramani, Kwonjoon Lee		Paper
AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion Mingzhen Sun, Weining Wang, Gen Li, Jiawei Liu, Jiahui Sun, Wanquan Feng, Shanshan Lao, SiYu Zhou, Qian He, Jing Liu		Paper

Biological and Drug Discovery

🔝 Back to Top

Title & Authors	Introduction	Links
MolEditRL: Structure-Preserving Molecular Editing via Discrete Diffusion and Reinforcement Learning Yuanxin Zhuang, Dazhong Shen, Ying Sun		Paper
CFP-Gen: Combinatorial Functional Protein Generation via Diffusion Language Models YJunbo Yin, Chao Zha, Wenjia He, Chencheng Xu, Xin Gao		Paper
Text-Guided Multi-Property Molecular Optimization with a Diffusion Language Model Yida Xiong, Kun Li, Jiameng Chen, Hongzhi Zhang, Di Lin, Yan Che, Wenbin Hu		Paper
GenMol: A Drug Discovery Generalist with Discrete Diffusion Seul Lee, Karsten Kreis, Srimukh Prasad Veccham, Meng Liu, Danny Reidenbach, Yuxing Peng, Saee Paliwal, Weili Nie, Arash Vahdat		Paper
DPLM-2: A Multimodal Diffusion Protein Language Model Xinyou Wang, Zaixiang Zheng, Fei Ye, Dongyu Xue, Shujian Huang, Quanquan Gu		Paper
PepTune: De Novo Generation of Therapeutic Peptides with Multi-Objective-Guided Discrete Diffusion Sophia Tang, Yinuo Zhang, Pranam Chatterjee		Paper

Toolkits

• Open-dLLM: https://github.com/pengzhangzhi/Open-dLLM
• dllm-trainer: https://github.com/ZHZisZZ/dllm-trainer/tree/main