Awesome Diffusion Models in Low-level Vision [](https://github.com/ChunmingHe/awesome-diffusion-models-in-low-level-vision)
February 23, 2025 · View on GitHub
🔥A curated list of awesome Diffusion Models(DMs) in low-level vision.🔥
Please feel free to offer your suggestions in the Issues and pull requests to add links.
[ Last updated at 2025/02/24 ]
Contents
Latest Works Recommended
Diffusion Models in Low-Level Vision: A Survey
Chunming He, Yuqi Shen, Chengyu Fang, Fengyang Xiao, Longxiang Tang, Yulun Zhang, Wangmeng Zuo, Zhenhua Guo, Xiu Li
TPAMI 2025. [Paper]
Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model
Chunming He, Chengyu Fang, Yulun Zhang, Kai Li, Longxiang Tang, Chenyu You, Fengyang Xiao, Zhenhua Guo, Xiu Li
ICLR 2025, Spotlight. [Paper] [Code]
Jan. 2025
Papers
1. Diffusion Models For Natural Image Restoration
1.1 General-purpose Image Restoration
1.1.1 Zero-shot DM-based IR
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/09 | DDPG | Image Restoration by Denoising Diffusion Models with Iteratively Preconditioned Guidance Tomer Garber,Tom Tirer | CVPR 2024 | Paper/Code |
| 2024/07 | DAVI | DAVI: Diffusion Prior-Based Amortized Variational Inference for Noisy Inverse Problems Sojin Lee, Dogyun Park, Inho Kong, Hyunwoo J. Kim | ECCV 2024 | Paper/Code |
| 2024/07 | - | Prototype Clustered Diffusion Models for Versatile Inverse Problems Jinghao Zhang, Zizheng Yang, Qi Zhu, Feng Zhao | arXiv 2024 | Paper/ |
| 2024/07 | ZAPS | Zero-Shot Adaptation for Approximate Posterior Sampling of Diffusion Models in Inverse Problems Yaşar Utku Alçalar, Mehmet Akçakaya | ECCV 2024 | Paper/Code |
| 2024/07 | DDIP3D | Deep Diffusion Image Prior for Efficient OOD Adaptation in 3D Inverse Problems Hyungjin Chung, Jong Chul Ye | ECCV 2024 | Paper/Code |
| 2024/07 | MoE-DiffIR | MoE-DiffIR: Task-customized Diffusion Priors for Universal Compressed Image Restoration Yulin Ren, Xin Li, Bingchen Li, Xingrui Wang, Mengxi Guo, Shijie Zhao, Li Zhang, Zhibo Chen | ECCV 2024 | Paper/Code |
| 2024/05 | DMPlug | DMPlug: A Plug-in Method for Solving Inverse Problems with Diffusion Models Hengkang Wang, Xu Zhang, Taihui Li, Yuxiang Wan, Tiancong Chen, Ju Sun | NeurIPS 2024 | Paper/Code |
| 2024/03 | Osmosis | Osmosis: RGBD Diffusion Prior for Underwater Image Restoration Opher Bar Nathan, Deborah Levy, Tali Treibitz, Dan Rosenbaum | ECCV 2024 | Paper/Code |
| 2024/03 | DCDP | Decoupled Data Consistency with Diffusion Purification for Image Restoration Xiang Li, Soo Min Kwon, Ismail R. Alkhouri, Saiprasad Ravishankar, Qing Qu | arXiv 2024 | Paper/Code |
| 2024/03 | Diff-Plugin | Diff-Plugin: Revitalizing Details for Diffusion-based Low-level Tasks Yuhao Liu, Zhanghan Ke, Fang Liu, Nanxuan Zhao, Rynson W.H. Lau | CVPR 2024 | Paper/Code |
| 2023/11 | DeqIR | Deep Equilibrium Diffusion Restoration with Parallel Sampling Jiezhang Cao, Yue Shi, Kai Zhang, Yulun Zhang, Radu Timofte, Luc Van Gool | CVPR 2024 | Paper/Code |
| 2023/09 | PGDiff | PGDiff: Guiding Diffusion Models for Versatile Face Restoration via Partial Guidance Peiqing Yang, Shangchen Zhou, Qingyi Tao, Chen Change Loy | NeurIPS 2023 | Paper/Code |
| 2023/05 | DiffPIR | Denoising diffusion models for plug-and-play image restoration Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, Luc Van Gool | CVPR 2023 | Paper/Code |
| 2023/05 | RED-Diff | A Variational Perspective on Solving Inverse Problems with Diffusion Models Morteza Mardani, Jiaming Song, Jan Kautz, Arash Vahdat | arXiv 2023 | Paper/ |
| 2023/04 | GDP | Generative Diffusion Prior for Unified Image Restoration and Enhancement Ben Fei, Zhaoyang Lyu, Liang Pan, Junzhe Zhang, Weidong Yang, Tianyue Luo, Bo Zhang, Bo Dai | CVPR 2023 | Paper/Code |
| 2023/04 | - | Score-Based Diffusion Models as Principled Priors for Inverse Imaging Berthy T. Feng, Jamie Smith, Michael Rubinstein, Huiwen Chang, Katherine L. Bouman, William T. Freeman | arXiv 2023 | Paper/ |
| 2023/03 | - | Unlimited-Size Diffusion Restoration Yinhuai Wang, Jiwen Yu, Runyi Yu, Jian Zhang | CVPR 2023 | Paper/Code |
| 2023/02 | πGDM | Pseudoinverse-Guided Diffusion Models for Inverse Problems Jiaming Song, Arash Vahdat, Morteza Mardani, Jan Kautz | ICLR 2023 | Paper/ |
| 2022/12 | ADIR | ADIR: Adaptive Diffusion for Image Reconstruction Shady Abu-Hussein, Tom Tirer, Raja Giryes | arXiv 2022 | Paper/ |
| 2022/12 | DDNM | Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model Yinhuai Wang, Jiwen Yu, Jian Zhang | ICLR 2023 | Paper/Code |
| 2022/09 | DPS | Diffusion Posterior Sampling for General Noisy Inverse Problems Hyungjin Chung, Jeongsol Kim, Michael T. Mccann, Marc L. Klasky, Jong Chul Ye | CVPR 2023 | Paper/Code |
| 2022/01 | MCG | Improving diffusion models for inverse problems using manifold constraints Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, Jong Chul Ye | NeurIPS 2022 | Paper/Code |
| 2022/01 | DDRM | Denoising Diffusion Restoration Models Bahjat Kawar, Michael Elad, Stefano Ermon, Jiaming Song | NeurIPS 2022 | Paper/Code |
| 2021/12 | CCDF | Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction Hyungjin Chung, Byeongsu Sim, Jong Chul Ye | CVPR 2022 | Paper/ |
| 2021/08 | ILVR | ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, Sungroh Yoon | ICCV 2021 | Paper/Code |
| 2021/05 | SNeurIPS | SNeurIPS: Solving Noisy Inverse Problems Stochastically Bahjat Kawar, Gregory Vaksman, Michael Elad | NeurIPS 2021 | Paper/Code |
1.1.2 Supervised DM-based IR
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/12 | VSPBFR | Visual Style Prompt Learning Using Diffusion Models for Blind Face Restoration Wanglong Lu, Jikai Wang, Tao Wang, Kaihao Zhang, Xianta Jiang, Hanli Zhao | Pattern Recognition 2025 | Paper/Code |
| 2024/10 | Resfusion | Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise Zhenning Shi, Haoshuai Zheng, Chen Xu, Changsheng Dong, Bin Pan, Xueshuo Xie, Along He, Tao Li, Huazhu Fu | NeurIPS 2024 | Paper/Code |
| 2024/09 | DTPM | Learning Diffusion Texture Priors for Image Restoration Tian Ye, Sixiang Chen, Wenhao Chai, Zhaohu Xing, Jing Qin, Ge Lin, Lei Zhu | CVPR 2024 | Paper/ |
| 2024/07 | Difface | Difface: Blind face restoration with diffused error contraction Zongsheng Yue, Chen Change Loy | TPAMI 2024 | Paper/Code |
| 2024/05 | AutoDIR | AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion Yitong Jiang, Zhaoyang Zhang, Tianfan Xue, Jinwei Gu | ECCV 2024 | Paper/Code |
| 2024/04 | SUPIR | Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild Fanghua Yu, Jinjin Gu, Zheyuan Li, Jinfan Hu, Xiangtao Kong, Xintao Wang, Jingwen He, Yu Qiao, Chao Dong | CVPR 2024 | Paper/Code |
| 2024/03 | MPerceiver | Multimodal Prompt Perceiver: Empower Adaptiveness, Generalizability and Fidelity for All-in-One Image Restoration Yuang Ai, Huaibo Huang, Xiaoqiang Zhou, Jiexiang Wang, Ran He | CVPR 2024 | Paper/ |
| 2024/03 | DiffUIR | Selective Hourglass Mapping for Universal Image Restoration Based on Diffusion Model Dian Zheng, Xiao-Ming Wu, Shuzhou Yang, Jian Zhang, Jian-Fang Hu, Wei-Shi Zheng | CVPR 2024 | Paper/Code |
| 2024/03 | - | Efficient Diffusion Model for Image Restoration by Residual Shifting Yuanzhi Zhu, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, Luc Van Gool | arXiv 2024 | Paper/Code |
| 2024/02 | WaveDM | WaveDM: Wavelet-Based Diffusion Models for Image Restoration Yi Huang, Jiancheng Huang, Jianzhuang Liu, Mingfu Yan, Yu Dong, Jiaxi Lv | IEEE Trans Multimedia 2024 | Paper/Code |
| 2023/11 | WF-Diff | Wavelet-based Fourier Information Interaction with Frequency Diffusion Adjustment for Underwater Image Restoration Chen Zhao, Weiling Cai, Chenyu Dong, Chengwei Hu | CVPR 2024 | Paper/Code |
| 2023/10 | Event-Diffusion | Event-Diffusion: Event-Based Image Reconstruction and Restoration with Diffusion Models Quanmin Liang, Xiawu Zheng, Kai Huang, Yan Zhang, Jie Chen, Yonghong Tian | MM 2023 | Paper/ |
| 2023/10 | C2F-DFT | Learning A Coarse-to-Fine Diffusion Transformer for Image Restoration Liyan Wang, Qinyu Yang, Cong Wang, Wei Wang, Jinshan Pan, Zhixun Su | arXiv 2023 | Paper/Code |
| 2023/08 | RDDM | Residual Denoising Diffusion Models Jiawei Liu, Qiang Wang, Huijie Fan, Yinong Wang, Yandong Tang, Liangqiong Qu | CVPR 2024 | Paper/Code |
| 2023/07 | PiRN | Physics-Driven Turbulence Image Restoration with Stochastic Refinement Ajay Jaiswal, Xingguang Zhang, Stanley H. Chan, Zhangyang Wang | ICCV 2023 | Paper/Code |
| 2023/08 | DiffBIR | DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior Xinqi Lin, Jingwen He, Ziyan Chen, Zhaoyang Lyu, Bo Dai, Fanghua Yu, Wanli Ouyang, Yu Qiao, Chao Dong | arXiv 2023 | Paper/Code |
| 2023/07 | SinDDM | SinDDM: A Single Image Denoising Diffusion Model Vladimir Kulikov, Shahar Yadin, Matan Kleiner, Tomer Michaeli | ICML 2023 | Paper/Code |
| 2023/07 | IDM | Towards Authentic Face Restoration with Iterative Diffusion Models and Beyond Yang Zhao, Tingbo Hou, Yu-Chuan Su, Xuhui Jia. Yandong Li, Matthias Grundmann | ICCV 2023 | Paper/ |
| 2023/05 | InDI | Inversion by Direct Iteration: An Alternative to Denoising Diffusion for Image Restoration Mauricio Delbracio, Peyman Milanfar | arXiv 2023 | Paper/Code |
| 2023/05 | UCDIR | A Unified Conditional Framework for Diffusion-based Image Restoration Yi Zhang, Xiaoyu Shi, Dasong Li, Xiaogang Wang, Jian Wang, Hongsheng Li | NeurIPS 2023 | Paper/Code |
| 2023/03 | DiracDiffusion | DiracDiffusion: Denoising and Incremental Reconstruction with Assured Data-Consistency Zalan Fabian, Berk Tinaz, Mahdi Soltanolkotabi | arXiv 2023 | Paper/ |
| 2023/04 | Refusion | Refusion: Enabling Large-Size Realistic Image Restoration with Latent-Space Diffusion Models Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao, Jens Sjölund, Thomas B. Schön | CVPRW 2023 | Paper/Code |
| 2023/01 | IR-SDE | Image Restoration with Mean-Reverting Stochastic Differential Equations Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao, Jens Sjölund, Thomas B. Schön | ICML 2023 | Paper/Code |
| 2021/12 | LDM | High-resolution image synthesis with latent diffusion models Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, Björn Ommer | CVPR 2022 | Paper/Code |
1.2 Task-specific Image Restoration
1.2.1 Super Resolution (SR)
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/12 | OFTSR | OFTSR: One-Step Flow for Image Super-Resolution with Tunable Fidelity-Realism Trade-offs Yuanzhi Zhu, Ruiqing Wang, Shilin Lu, Junnan Li, Hanshu Yan, Kai Zhang | arXiv 2024 | Paper/Code |
| 2024/09 | SinSR | SinSR: Diffusion-Based Image Super-Resolution in a Single Step Yufei Wang, Wenhan Yang, Xinyuan Chen, Yaohui Wang, Lanqing Guo, Lap-Pui Chau, Ziwei Liu, Yu Qiao, Alex C. Kot, Bihan Wen | CVPR 2024 | Paper/Code |
| 2024/09 | CCSR | Improving the Stability and Efficiency of Diffusion Models for Content Consistent Super-Resolution Lingchen Sun, Rongyuan Wu, Jie Liang, Zhengqiang Zhang, Hongwei Yong, Lei Zhang | arXiv 2024 | Paper/Code |
| 2024/07 | DiWa | Waving goodbye to low-res: A diffusion-wavelet approach for image super-resolution Brian B. Moser, Stanislav Frolov, Federico Raue, Sebastian Palacio, Andreas Dengel | IJCNN 2024 | Paper/ |
| 2024/05 | CDFormer | CDFormer:When Degradation Prediction Embraces Diffusion Model for Blind Image Super-Resolution Qingguo Liu, Chenyi Zhuang, Pan Gao, Jie Qin | CVPR 2024 | Paper/Code |
| 2024/4 | OmniSSR | OmniSSR: Zero-shot Omnidirectional Image Super-Resolution using Stable Diffusion Model Runyi Li, Xuhan Sheng, Weiqi Li, Jian Zhang | ECCV 2024 | Paper/Code |
| 2024/04 | DiffMSR | Rethinking Diffusion Model for Multi-Contrast MRI Super-Resolution Guangyuan Li, Chen Rao, Juncheng Mo, Zhanjie Zhang, Wei Xing, Lei Zhao | CVPR 2024 | Paper/Code |
| 2024/04 | DiSR-NeRF | DiSR-NeRF: Diffusion-Guided View-Consistent Super-Resolution NeRF Jie Long Lee, Chen Li, Gim Hee Lee | CVPR 2024 | Paper/Code |
| 2024/03 | RefDiff | Building Bridges across Spatial and Temporal Resolutions: Reference-Based Super-Resolution via Change Priors and Conditional Diffusion Model Runmin Dong, Shuai Yuan, Bin Luo, Mengxuan Chen, Jinxiao Zhang, Lixian Zhang, Weijia Li, Juepeng Zheng, Haohuan Fu | CVPR 2024 | Paper/Code |
| 2024/03 | SARGD | Self-Adaptive Reality-Guided Diffusion for Artifact-Free Super-Resolution -- | CVPR 2024 | Paper/Code |
| 2024/03 | - | Arbitrary-Scale Image Generation and Upsampling using Latent Diffusion Model and Implicit Neural Decoder Jinseok Kim, Tae-Kyun Kim | CVPR 2024 | Paper/Code |
| 2024/03 | XPSR | XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution Yunpeng Qu, Kun Yuan, Kai Zhao, Qizhi Xie, Jinhua Hao, Ming Sun, Chao Zhou | ECCV 2024 | Paper/Code |
| 2024/02 | SAM-DiffSR | SAM-DiffSR: Structure-Modulated Diffusion Model for Image Super-Resolution Chengcheng Wang, Zhiwei Hao, Yehui Tang, Jianyuan Guo, Yujie Yang, Kai Han, Yunhe Wang | arXiv 2024 | Paper/Code |
| 2024/01 | SeeSR | SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution Rongyuan Wu, Tao Yang, Lingchen Sun, Zhengqiang Zhang, Shuai Li, Lei Zhang | CVPR 2024 | Paper/Code |
| 2023/12 | DiffTSR | Diffusion-based Blind Text Image Super-Resolution Yuzhe Zhang, Jiawei Zhang, Hao Li, Zhouxia Wang, Luwei Hou, Dongqing Zou, Liheng Bian | CVPR 2024 | Paper/Code |
| 2023/12 | stisr-tcdm | Scene Text Image Super-resolution based on Text-conditional Diffusion Models Chihiro Noguchi, Shun Fukuda, Masao Yamanaka | WACV 2024 | Paper/Code |
| 2023/11 | CoSeR | CoSeR: Bridging Image and Language for Cognitive Super-Resolution Haoze Sun, Wenbo Li, Jianzhuang Liu, Haoyu Chen, Renjing Pei, Xueyi Zou, Youliang Yan, Yujiu Yang | CVPR 2024 | Paper/Code |
| 2023/10 | MoESR | Image Super-resolution Via Latent Diffusion: A Sampling-space Mixture Of Experts And Frequency-augmented Decoder Approach Feng Luo, Jinxi Xiang, Jun Zhang, Xiao Han, Wei Yang | arXiv 2023 | Paper/Code |
| 2023/09 | - | License Plate Super-Resolution Using Diffusion Models Sawsan AlHalawani, Bilel Benjdira, Adel Ammar, Anis Koubaa, Anas M. Ali | arXiv 2023 | Paper/ |
| 2023/08 | PASD | Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization Tao Yang, Rongyuan Wu, Peiran Ren, Xuansong Xie, Lei Zhang | arXiv 2023 | Paper/Code |
| 2023/07 | ResShift | ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting Zongsheng Yue, Jianyi Wang, Chen Change Loy | NeurIPS 2023 | Paper/Code |
| 2023/07 | PartDiff | PartDiff: Image Super-resolution with Partial Diffusion Models Axi Niu, Pham Xuan Trung, Kang Zhang, Jinqiu Sun, Yu Zhu, In So Kweon, Yanning Zhang | arXiv 2023 | Paper/ |
| 2023/07 | ACDMSR | ACDMSR: Accelerated Conditional Diffusion Models for Single Image Super-Resolution Axi Niu, Pham Xuan Trung, Kang Zhang, Jinqiu Sun, Yu Zhu, In So Kweon, Yanning Zhang | IEEE Trans. Broadcast. 2024 | Paper/ |
| 2023/05 | StableSR | Exploiting Diffusion Prior for Real-World Image Super-Resolution Jianyi Wang, Zongsheng Yue, Shangchen Zhou, Kelvin C.K. Chan, Chen Change Loy | arXiv 2023 | Paper/Code |
| 2023/03 | DR2 | DR2: Diffusion-Based Robust Degradation Remover for Blind Face Restoration Zhixin Wang, Xiaoyun Zhang, Ziying Zhang, Huangjie Zheng, Mingyuan Zhou, Ya Zhang, Yanfeng Wang | CVPR 2023 | Paper/Code |
| 2023/03 | ResDiff | ResDiff: Combining CNN and Diffusion Model for Image Super-Resolution Shuyao Shang, Zhengyang Shan, Guangxing Liu, Jinglin Zhang | AAAI 2024 | Paper/Code |
| 2023/03 | IDM | Implicit Diffusion Models for Continuous Super-Resolution Sicheng Gao, Xuhui Liu, Bohan Zeng, Sheng Xu, Yanjing Li, Xiaoyan Luo, Jianzhuang Liu, Xiantong Zhen, Baochang Zhang | CVPR 2023 | Paper/Code |
| 2023/02 | - | Denoising Diffusion Probabilistic Models for Robust Image Super-Resolution in the Wild Hshmat Sahak, Daniel Watson, Chitwan Saharia, David Fleet | arXiv 2023 | Paper/ |
| 2023/02 | CDPMSR | CDPMSR: Conditional Diffusion Probabilistic Models for Single Image Super-Resolution Axi Niu, Kang Zhang, Trung X. Pham, Jinqiu Sun, Yu Zhu, In So Kweon, Yanning Zhang | arXiv 2023 | Paper/ |
| 2022/09 | SUE-SR | Face Super-Resolution Using Stochastic Differential Equations Marcelo dos Santos, Rayson Laroca, Rafael O. Ribeiro, João Neves, Hugo Proença, David Menotti | SIGGRAPH 2022 | Paper/Code |
| 2021/05 | CDM | Cascaded Diffusion Models for High Fidelity Image Generation Jonathan Ho, Chitwan Saharia, William Chan, David J. Fleet, Mohammad Norouzi, Tim Salimans. | JMLR 2022 | Paper/Code |
| 2021/04 | SR3 | Image Super-Resolution via Iterative Refinement Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J. Fleet, Mohammad Norouzi. | TPAMI 2022 | Paper/Code |
| 2021/04 | SRDiff | SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models H. Li, Y. Yang, M. Chang, S. Chen, H. Feng, Z. Xu, Q. Li, and Y. Chen. | Neurocomputing 2022 | Paper/Code |
1.2.2 Inpainting
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/09 | CAT-Diffusion | Improving Text-guided Object Inpainting with Semantic Pre-inpainting Yifu Chen, Jingwen Chen, Yingwei Pan, Yehao Li, Ting Yao, Zhineng Chen, Tao Mei | ECCV 2024 | Paper/Code |
| 2024/08 | GSDM | Text Image Inpainting via Global Structure-Guided Diffusion Models Shipeng Zhu, Pengfei Fang, Chenjie Zhu, Zuoyan Zhao, Qiang Xu, Hui Xue | AAAI 2024 | Paper/Code |
| 2024/07 | PILOT | Coherent and Multi-modality Image Inpainting via Latent Space Optimization Lingzhi Pan, Tong Zhang, Bingyuan Chen, Qi Zhou, Wei Ke, Sabine Süsstrunk, Mathieu Salzmann | arXiv 2024 | Paper/Code |
| 2024/07 | Diffree | Diffree: Text-Guided Shape Free Object Inpainting with Diffusion Model Lirui Zhao, Tianshuo Yang, Wenqi Shao, Yuxin Zhang, Yu Qiao, Ping Luo, Kaipeng Zhang, Rongrong Ji | arxiv 2024 | Paper/Code |
| 2024/03 | StrDiffusion | Structure Matters: Tackling the Semantic Discrepancy in Diffusion Models for Image Inpainting Haipeng Liu, Yang Wang, Biao Qian, Meng Wang, Yong Rui | CVPR 2024 | Paper/Code |
| 2024/03 | MMGInpainting | MMGInpainting: Multi-Modality Guided Image Inpainting Based On Diffusion Models Cong Zhang, Wenxia Yang, Xin Li, Huan Han | IEEE Trans Multimedia 2024 | Paper/Code |
| 2024/03 | BrushNet | BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion Xuan Ju, Xian Liu, Xintao Wang, Yuxuan Bian, Ying Shan, Qiang Xu | arXiv 2024 | Paper/Code |
| 2024/03 | - | Fill in the ____ (a Diffusion-based Image Inpainting Pipeline) Eyoel Gebre, Krishna Saxena, Timothy Tran | arXiv 2024 | Paper/ |
| 2024/01 | LatentPaint | LatentPaint: Image Inpainting in Latent Space With Diffusion Models Ciprian Corneanu, Raghudeep Gadde, Aleix M. Martinez | WACV 2024 | Paper/ |
| 2023/12 | - | Amodal Completion via Progressive Mixed Context Diffusion Katherine Xu, Lingzhi Zhang, Jianbo Shi | CVPR 2024 | Paper/Code |
| 2023/11 | Tiramisu | Image Inpainting via Tractable Steering of Diffusion Models Anji Liu, Mathias Niepert, Guy Van den Broeck | arXiv 2023 | Paper/Code |
| 2023/10 | Uni-paint | Uni-paint: A Unified Framework for Multimodal Image Inpainting with Pretrained Diffusion Model Shiyuan Yang, Xiaodong Chen, Jing Liao | MM 2023 | Paper/Code |
| 2023/09 | Gradpaint | Gradpaint: Gradient-Guided Inpainting with Diffusion Models Asya Grechka, Guillaume Couairon, Matthieu Cord | CVIU 2024 | Paper/Code |
| 2022/12 | SmartBrush | SmartBrush: Text and Shape Guided Object Inpainting With Diffusion Model Shaoan Xie 1, Zhifei Zhang, Zhe Lin, Tobias Hinz, Kun Zhang* | CVPR 2023 | Paper/ |
| 2023/04 | Copaint | Towards Coherent Image Inpainting Using Denoising Diffusion Implicit Models Guanhua Zhang, Jiabao Ji, Yang Zhang, Mo Yu, Tommi Jaakkola, Shiyu Chang | ICML 2023 | Paper/Code |
| 2022/05 | Palette | Palette: Image-to-image diffusion models C. Saharia, W. Chan, H. Chang, C. Lee, J. Ho, T. Salimans, D. Fleet, and M. Norouzi. | SIGGRAPH 2022 | Paper/Code |
| 2022/01 | RePaint | Repaint: Inpainting using denoising diffusion probabilistic models A. Lugmayr, M. Danelljan, A. Romero, F. Yu, R. Timofte, and L. Van Gool. | CVPR 2022 | Paper/Code |
1.2.3 Deblur
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/04 | Diffevent | Diffevent: Event Residual Diffusion for Image Deblurring Pei Wang, Jiumei He, Qingsen Yan, Yu Zhu, Jinqiu Sun, Yanning Zhang | ICASSP 2024 | Paper/Code |
| 2024/01 | FastDiffusionEM | Fast Diffusion EM: a diffusion model for blind inverse problems with application to deconvolution Charles Laroche, Andrés Almansa, Eva Coupete | WACV 2024 | Paper/Code |
| 2024/01 | SI-DDPM-FMO | Single-Image Deblurring, Trajectory and Shape Recovery of Fast Moving Objects With Denoising Diffusion Probabilistic Models Radim Spetlik, Denys Rozumnyi, Jiřà Matas | WACV 2024 | Paper/Code |
| 2023/12 | ID-Blau | ID-Blau: Image Deblurring by Implicit Diffusion-based reBLurring AUgmentation Jia-Hao Wu, Fu-Jen Tsai, Yan-Tsung Peng, Chung-Chi Tsai, Chia-Wen Lin, Yen-Yu Lin | CVPR 2024 | Paper/Code |
| 2023/05 | HI-Diff | Hierarchical Integration Diffusion Model for Realistic Image Deblurring Zheng Chen, Yulun Zhang, Ding Liu, Bin Xia, Jinjin Gu, Linghe Kong, Xin Yuan | arXiv 2023 | Paper/ |
| 2022/12 | - | Multiscale Structure Guided Diffusion for Image Deblurring M. Ren, M. Delbracio, H. Talebi, G. Gerig, and P. Milanfar. | ICCV 2023 | Paper/ |
| 2021/12 | DVSR | Deblurring via Stochastic Refinement Jay Whang, Mauricio Delbracio, Hossein Talebi, Chitwan Saharia, Alexandros G. Dimakis, Peyman Milanfar | CVPR 2022 | Paper/Code |
1.2.4 Dehaze
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/09 | DiffLI2D | Unleashing the Potential of the Semantic Latent Space in Diffusion Models for Image Dehazing Zizheng Yang, Hu Yu, Bing Li, Jinghao Zhang, Jie Huang, Feng Zhao | ECCV 2024 | Paper/ |
| 2024/08 | - | Image Dehazing Method Based on Diffusion Model Fengxu Guan; Haitao Lai; Hanyu Zang; Jinbao Huang | ICMA 2024 | Paper/ |
| 2024/08 | MP-DDPM | A Multi-scale Patch Approach with Diffusion Model for Image Dehazing Yao Guo, Yongliang Wu & Changsheng Wan | ICIC 2024 | Paper/ |
| 2024/07 | FP-Diff | Frequency-based and Physics-guiding Diffusion Model for Single Image Dehazing Siying Xie; Fuping Li; Mingye Ju | CCC 2024 | Paper/ |
| 2024/07 | DehazeDiff | DehazeDiff: When Conditional Guidance Meets Diffusion Models for Image Dehazing Longyu Cheng, Xujin Ba, Yanyun Qu | ISCAS 2024 | Paper/ |
| 2024/05 | - | Image Dehazing based on Iterative-Refining Diffusion Model Jiarong Wang, Hao Hu | ICIGP 2024 | Paper/ |
| 2024/04 | JCDM | Joint Conditional Diffusion Model for Image Restoration with Mixed Degradations Yufeng Yue, Meng Yu, Luojie Yang, Yi Yang | arXiv 2024 | Paper/Code |
| 2024/04 | DehazeDDPM | High-quality Image Dehazing with Diffusion Model Hu Yu, Jie Huang, Kaiwen Zheng, Feng Zhao | arXiv 2024 | Paper/Code |
| 2023/10 | DehazeDM | DehazeDM: Image Dehazing via Patch Autoencoder Based on Diffusion Models Yuming Yang; Dongsheng Zou; Xinyi Song; Xiaotong Zhang | SMC 2023 | Paper |
| 2023/08 | HazeAug | Frequency Compensated Diffusion Model for Real-scene Dehazing Jing Wang, Songtao Wu, Kuanhong Xu, Zhiqiang Yuan | Neural Networks 2024 | Paper/Code |
| 2022/11 | WeatherDiff | Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models Ozan Özdenizci, Robert Legenstein | TPAMI 2023 | Paper/Code |
1.2.5 Low-light Enhancement
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2025/01 | Reti-Diff | Reti-Diff: Illumination Degradation Image Restoration with Retinex-based Latent Diffusion Model Chunming He, Chengyu Fang, Yulun Zhang, Longxiang Tang, Jinfa Huang, Kai Li, Zhenhua Guo, Xiu Li, Sina Farsiu | ICLR 2025 | Paper/Code |
| 2024/10 | - | Super-resolving Real-world Image Illumination Enhancement: A New Dataset and A Conditional Diffusion Model Yang Liu, Yaofang Liu, Jinshan Pan, Yuxiang Hui, Fan Jia, Raymond H. Chan, Tieyong Zeng | arXiv 2024 | Paper/Code |
| 2024/09 | FourierDiff | Fourier Priors-Guided Diffusion for Zero-Shot Joint Low-Light Enhancement and Deblurring Xiaoqian Lv, Shengping Zhang, Chenyang Wang, Yichen Zheng, Bineng Zhong, Chongyi Li | CVPR 2024 | Paper/Code |
| 2024/09 | DiffLight | DiffLight: Integrating Content and Detail for Low-light Image Enhancement Yixu Feng, Shuo Hou, Haotian Lin, Yu Zhu, Peng Wu, Wei Dong, Jinqiu Sun, Qingsen Yan, Yanning Zhang | CVPR 2024 | Paper/ |
| 2024/09 | DLDiff | DLDiff: Image Detail-guided Latent Diffusion Model for Low-Light Image Enhancement Minglong Xue; Yanyi He; Jinhong He; Senming Zhong | LSP 2024 | Paper/Code |
| 2024/09 | - | Low Light Image Enhancement Based on a Pyramid Diffusion Model Xiang Li; Chunling Liu; Hui Cheng | ISPDS 2024 | Paper/ |
| 2024/08 | - | Image Intrinsic Components Guided Conditional Diffusion Model for Low-light Image Enhancement Sicong Kang; Shuaibo Gao; Wenhui Wu; Xu Wang; Shuoyao Wang; Guoping Qiu | TCSVT 2024 | Paper/ |
| 2024/08 | MDDE | Low Light Image Enhancement Based on Retinex Theory and Diffusion Model Tao Chen, Dongmei Liu | ICDSP 2024 | Paper/ |
| 2024/07 | PSC diffusion | PSC diffusion: patch-based simplified conditional diffusion model for low-light image enhancement Fei Wan, Bingxin Xu, Weiguo Pan, Hongzhe Liu | Multimed. Syst. 2024 | Paper/ |
| 2024/07 | JoReS-Diff | JoReS-Diff: Joint Retinex and Semantic Priors in Diffusion Model for Low-light Image Enhancement Yuhui Wu, Guoqing Wang, Zhiwen Wang, Yang Yang, Tianyu Li, Malu Zhang, Chongyi Li, Heng Tao Shen | MM 2024 | Paper/ |
| 2024/07 | AGLLDiff | AGLLDiff: Guiding Diffusion Models Towards Unsupervised Training-free Real-world Low-light Image Enhancement Yunlong Lin, Tian Ye, Sixiang Chen, Zhenqi Fu, Yingying Wang, Wenhao Chai, Zhaohu Xing, Lei Zhu, Xinghao Ding | arXiv 2024 | Paper/Code |
| 2024/07 | LightenDiffusion | LightenDiffusion: Unsupervised Low-Light Image Enhancement with Latent-Retinex Diffusion Models Hai Jiang, Ao Luo, Xiaohong Liu, Songchen Han, Shuaicheng Liu | ECCV 2024 | Paper/Code |
| 2024/07 | Zero-LED | Zero-LED: Zero-Reference Lighting Estimation Diffusion Model for Low-Light Image Enhancement Jinhong He, Minglong Xue, Aoxiang Ning, Chengyun Song | arXiv 2024 | Paper/ |
| 2024/07 | - | Enhancing Low-Light Images: A Novel Approach Combining Anisotropic Diffusion and Retinex Mingyang Sun; Ru Yi; Xinxin Wang; Ningtao Ma | CSCWD 2024 | Paper/ |
| 2024/07 | SVBoost | Low Light Enhancement in Street Scenes Based on Diffusion Model Rui Xia; Lisong Wang; Taili Li; Pingping Shi | CSCWD 2024 | Paper/ |
| 2024/04 | LightDiff | Light the Night: A Multi-Condition Diffusion Framework for Unpaired Low-Light Enhancement in Autonomous Driving Jinlong Li, Baolu Li, Zhengzhong Tu, Xinyu Liu, Qing Guo, Felix Juefei-Xu, Runsheng Xu, Hongkai Yu | CVPR 2024 | Paper/Code |
| 2024/04 | - | A ground-based dataset and a diffusion model for on-orbit low-light image enhancement Yiman Zhu, Lu Wang, Jingyi Yuan, Yu Guo | arXiv 2024 | Paper/ |
| 2024/04 | SG-DDPM | SG-DDPM: Semantic-Guided Diffusion Model for Low-Light Image Enhancement Shize Wang | ICCECE 2024 | Paper/ |
| 2024/03 | MDMS | Multi-Domain Multi-Scale Diffusion Model for Low-Light Image Enhancement Kai Shang, Mingwen Shao, Chao Wang, Yuanshuo Cheng, Shuigen Wang | AAAI 2024 | Paper/Code |
| 2024/02 | TDS | TDS: Two Diffusion Streams for Low-Light Image Enhancement Jieming Wang; Xianqin Liu; Yijun Zhang; Jianfang Hu | CSECS 2024 | Paper/ |
| 2024/01 | CFWD | Low-light Image Enhancement via CLIP-Fourier Guided Wavelet Diffusion Minglong Xue, Jinhong He, Yanyi He, Zhipu Liu, Wenhai Wang, Mingliang Zhou | arXiv 2024 | Paper/Code/ |
| 2023/12 | L2DM | L2DM: A Diffusion Model for Low-Light Image Enhancement Lv, Xingguo and Dong, Xingbo and Jin, Zhe and Zhang, Hui and Song, Siyi and Li, Xuejun | PRCV 2023 | Paper/Code |
| 2023/10 | GSAD | Global Structure-Aware Diffusion Process for Low-Light Image Enhancement Jinhui Hou, Zhiyu Zhu, Junhui Hou, Hui Liu, Huanqiang Zeng, Hui Yuan | NeurIPS 2023 | Paper/Code |
| 2023/10 | CLEDiffusion | CLE Diffusion: Controllable Light Enhancement Diffusion Model Yuyang Yin, Dejia Xu, Chuangchuang Tan, Ping Liu, Yao Zhao, Yunchao Wei | MM 2023 | Paper/Code |
| 2023/10 | LLDE | LLDE: Enhancing Low-Light Images with Diffusion Model Xin Peng Oo, Chee Seng Chan | ICIP 2023 | Paper/Code |
| 2023/09 | - | Bootstrap Diffusion Model Curve Estimation for High Resolution Low-Light Image Enhancement Jiancheng Huang, Yifan Liu, Shifeng Chen | PRICAI 2023 | Paper/ |
| 2023/08 | DiffLLE | DiffLLE: Diffusion-guided Domain Calibration for Unsupervised Low-light Image Enhancement Shuzhou Yang, Xuanyu Zhang, Yinhuai Wang, Jiwen Yu, Yuhan Wang, Jian Zhang | arXiv 2023 | Paper/ |
| 2023/08 | ExposureDiffusion | ExposureDiffusion: Learning to Expose for Low-light Image Enhancement Yufei Wang, Yi Yu, Wenhan Yang, Lanqing Guo, Lap-Pui Chau, Alex C. Kot, Bihan Wen | ICCV 2023 | Paper/Code |
| 2023/08 | Diff-Retinex | Diff-Retinex: Rethinking Low-light Image Enhancement with A Generative Diffusion Model Xunpeng Yi, Han Xu, Hao Zhang, Linfeng Tang, Jiayi Ma | ICCV 2023 | Paper/Code |
| 2023/08 | CLE Diffusion | CLE Diffusion: Controllable Light Enhancement Diffusion Model Yuyang Yin, Dejia Xu, Chuangchuang Tan, Ping Liu, Yao Zhao, Yunchao Wei | MM 23 | Paper/Code |
| 2023/07 | LLDiffusion | LLDiffusion: Learning Degradation Representations in Diffusion Models for Low-Light Image Enhancement Tao Wang, Kaihao Zhang, Ziqian Shao, Wenhan Luo, Bjorn Stenger, Tae-Kyun Kim, Wei Liu, Hongdong Li | arXiv 2023 | Paper/Code |
| 2023/07 | DiffLIE | DiffLIE: Low-Light Image Enhancment based on Deep Diffusion Model Guanyu Wu; Cheng. Jin | ISCTIS 2023 | Paper/ |
| 2023/06 | - | Diffusion Model Based Low-Light Image Enhancement for Space Satellite Yiman Zhu, Lu Wang, Jingyi Yuan, Yu Guo | arXiv 2023 | Paper/ |
| 2023/05 | PyDiff | Pyramid Diffusion Models For Low-light Image Enhancement Dewei Zhou, Zongxin Yang, Yi Yang | IJCAI 2023 | Paper/Code |
| 2023/03 | LPDM | Denoising Diffusion Post-Processing for Low-Light Image Enhancement Savvas Panagiotou, Anna S. Bosman | arXiv 2023 | Paper/Code |
| 2023/03 | DiD | Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognition Cindy M. Nguyen, Eric R. Chan, Alexander W. Bergman, Gordon Wetzstein | WACV 2024 | Paper/Code |
| 2023/01 | DiffLL | Low-Light Image Enhancement with Wavelet-based Diffusion Models Hai Jiang, Ao Luo, Songchen Han, Haoqiang Fan, Shuaicheng Liu | TOG 2023 | Paper/Code |
1.2.6 Image Fusion
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/12 | MADMFuse | MADMFuse: A multi-attribute diffusion model to fuse infrared and visible images Hang Xu, Rencan Nie, Jinde Cao, Mingchuan Tan, Zhengze Ding | DSP 2024 | Paper/ |
| 2024/10 | Diff-IF | Diff-IF: Multi-modality image fusion via diffusion model with fusion knowledge prior Xunpeng Yi, Linfeng Tang, Hao Zhang, Han Xu, Jiayi Ma | Inf. Fusion 2024 | Paper/Code |
| 2024/08 | LFDT-Fusion | LFDT-Fusion: A latent feature-guided diffusion Transformer model for general image fusion Bo Yang, Zhaohui Jiang, Dong Pan, Haoyang Yu, Gui Gui, Weihua Gui | Inf. Fusion 2025 | Paper/Code |
| 2024/07 | FusionDiff | FusionDiff: A unified image fusion network based on diffusion probabilistic models Zefeng Huang, Shen Yang , Jin Wu, Lei Zhu, Jin Liu | CVIU 2024 | Paper/ |
| 2024/07 | GLAD | GLAD: A Global-Attention-Based Diffusion Model for Infrared and Visible Image Fusion Haozhe Guo, Mengjie Chen, Kaijiang Li, Hao Su, and Pei Lv | ICIC 2024 | Paper/ |
| 2024/04 | Dif-PAN | Diffusion model with disentangled modulations for sharpening multispectral and hyperspectral images Zihan Cao, Shiqi Cao, Liang-Jian Deng, Xiao Wu, Junming Hou, Gemine Vivone | Inf. Fusion 2024 | Paper/Code |
| 2023/06 | FusionDiff | FusionDiff: Multi-focus image fusion using denoising diffusion probabilistic models Mining Li, Ronghao Pei, Tianyou Zheng, Yang Zhang, Weiwei Fu | ESWA 2024 | Paper/Code |
| 2023/04 | DDRF | DDRF: Denoising Diffusion Model for Remote Sensing Image Fusion ZiHan Cao, ShiQi Cao, Xiao Wu, JunMing Hou, Ran Ran, Liang-Jian Deng | arXiv 2023 | Paper/ |
| 2023/03 | DDFM | DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion Zixiang Zhao, Haowen Bai, Yuanzhi Zhu, Jiangshe Zhang, Shuang Xu, Yulun Zhang, Kai Zhang, Deyu Meng, Radu Timofte, Luc Van Gool | ICCV 2023 | Paper/Code |
| 2023/01 | Dif-Fusion | Dif-Fusion: Towards High Color Fidelity in Infrared and Visible Image Fusion with Diffusion Models Jun Yue, Leyuan Fang, Shaobo Xia, Yue Deng, Jiayi Ma | TIP 2023 | Paper/Code |
2. Extended Diffusion Models In Low-level Vision
2.1 Diffusion Models In Low-level Medical Image Analysis
2.1.1 MRI
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/08 | DiMO | Diffusion Modeling with Domain-conditioned Prior Guidance for Accelerated MRI and qMRI Reconstruction Wanyu Bian, Albert Jang, Liping Zhang, Xiaonan Yang, Zachary Stewart, Fang Liu | TMI 2024 | Paper/ |
| 2024/04 | DiffMSR | Rethinking Diffusion Model for Multi-Contrast MRI Super-Resolution Guangyuan Li, Chen Rao, Juncheng Mo, Zhanjie Zhang, Wei Xing, Lei Zhao | CVPR 2024 | Paper/Code |
| 2024/05 | PPN | Fast Controllable Diffusion Models for Undersampled MRI Reconstruction Wei Jiang, Zhuang Xiong, Feng Liu, Nan Ye, Hongfu Sun | ISIB 2024 | Paper/ |
| 2024/02 | MRPD | MRPD: Undersampled MRI reconstruction by prompting a large latent diffusion model Ziqi Gao, S. Kevin Zhou | arxiv 2024 | Paper/Code |
| 2024/01 | HFS-SDE | High-Frequency Space Diffusion Model for Accelerated MRI Chentao Cao, Zhuo-Xu Cui, Yue Wang, Shaonan Liu, Taiin Chen, Hairong Zheng | TMI 2024 | Paper/Code |
| 2023/10 | SMRD | SMRD: SURE-Based Robust MRI Reconstruction with Diffusion Models Batu Ozturkler, Chao Liu, Benjamin Eckart, Morteza Mardani, Jiaming Song, Jan Kautz | MICCAI 2023 | Paper/Code |
| 2023/10 | InverseSR | InverseSR: 3D Brain MRI Super-Resolution Using a Latent Diffusion Model Jueqi Wang, Jacob Levman, Walter Hugo Lopez Pinaya, Petru-Daniel Tudosiu, M. Jorge Cardoso & Razvan Marinescu | MICCAI 2023 | Paper/Code |
| 2023/10 | DisC-Diff | DisC-Diff: Disentangled Conditional Diffusion Model for Multi-contrast MRI Super-Resolution Ye Mao, Lan Jiang, Xi Chen, Chao Li | MICCAI 2023 | Paper/Code |
| 2023/10 | CDiffMR | CDiffMR: Can We Replace the Gaussian Noise with K-Space Undersampling for Fast MRI? Jiahao Huang, Angelica I. Aviles-Rivero, Carola-Bibiane Schönlieb, Guang Yang | MICCAI 2023 | Paper/Code |
| 2023/10 | SSDiffRecon | Self-supervised MRI Reconstruction with Unrolled Diffusion Models Yilmaz Korkmaz, Tolga Cukur, Vishal M. Patel | MICCAI 2023 | Paper/Code |
| 2023/08 | - | Super-resolution of brain MRI images based on denoising diffusion probabilistic model Zhanxiong Wu, Xuanheng Chen, Sangma Xie, Jian Shen, Yu Zeng | BSPC 2023 | Paper/ |
| 2023/03 | - | Bayesian MRI reconstruction with joint uncertainty estimation using diffusion models Guanxiong Luo, Moritz Blumenthal, Martin Heide, Martin Uecker | MRM 2023 | Paper/ |
| 2022/09 | MC-DDPM | Measurement-Conditioned Denoising Diffusion Probabilistic Model for Under-Sampled Medical Image Reconstruction Yutong Xie, Quanzheng Li | MICCAI 2022 | Paper/Code |
| 2022/09 | DiffuseRecon | Towards Performant and Reliable Undersampled MR Reconstruction via Diffusion Model Sampling Cheng Peng, Pengfei Guo, S. Kevin Zhou, Vishal M. Patel & Rama Chellappa | MICCAI 2022 | Paper/Code |
| 2022/07 | AdaDiff | Adaptive Diffusion Priors for Accelerated MRI Reconstruction Alper Güngör, Salman UH Dar, Şaban Öztürk, Yilmaz Korkmaz, Gokberk Elmas, Muzaffer Özbey, Tolga Çukur | MedIA 2023 | Paper/Code |
| 2021/10 | Score-MRI | Score-based diffusion models for accelerated MRI Hyungjin Chung, Jong Chul Ye | MedIA 2022 | Paper/Code |
| 2021/08 | CSGM | Robust Compressed Sensing MRI with Deep Generative Priors Alper Güngör, Salman UH Dar, Şaban Öztürk, Yilmaz Korkmaz, Gokberk Elmas, Muzaffer Özbey, Tolga Çukur | NeurIPS 2021 | Paper/Code |
2.1.2 X-ray-based
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/10 | - | Diffusion-Based Generative Image Outpainting for Recovery of FOV-Truncated CT Images Michelle Espranita Liman, Daniel Rueckert, Florian J. Fintelmann, Philip Müller | MICCAI 2024 | Paper/Code |
| 2024/10 | PrideDiff | PrideDiff: Physics-Regularized Generalized Diffusion Model for CT Reconstruction Zexin Lu; Qi Gao; Tao Wang; Ziyuan Yang; Zhiwen Wang; Hui Yu | TRPMS 2024 | Paper/Code |
| 2024/10 | - | Ultrafast Short-Arc Diffusion-Based Cone Beam CT Image Reconstruction Y. FU, H. Zhang, W. Cai, L. Kuo, H. Xie, J.J. Cuaron, L.I. Cervino, J.M. Moran, X. Li1, T. Li | IJROBP 2024 | Paper/ |
| 2024/09 | RN-SDEs | RN-SDEs: Limited-Angle CT Reconstruction with Residual Null-Space Diffusion Stochastic Differential Equations Jiaqi Guo, Santiago Lopez-Tapia, Wing Shun Li, Yunnan Wu, Marcelo Carignano, Vadim Backman, Vinayak P. Dravid, Aggelos K. Katsaggelos | arXiv 2024 | Paper/ |
| 2024/09 | CT-SDM | CT-SDM: A Sampling Diffusion Model for Sparse-View CT Reconstruction across All Sampling Rates Liutao Yang, Jiahao Huang, Guang Yang, Daoqiang Zhang | arXiv 2024 | Paper/ |
| 2024/08 | - | CT reconstruction using diffusion posterior sampling conditioned on a nonlinear measurement model Shudong Li, Xiao Jiang, Matthew Tivnan, Grace J. Gang, Yuan Shen, J. Webster Stayman | SPIE 2024 | Paper/ |
| 2024/08 | DIFR3CT | DIFR3CT: Latent Diffusion for Probabilistic 3D CT Reconstruction from Few Planar X-Rays Yiran Sun, Hana Baroudi, Tucker Netherton, Laurence Court, Osama Mawlawi, Ashok Veeraraghavan, Guha Balakrishnan | arXiv 2024 | Paper/Code |
| 2024/08 | FCDM | FCDM: Sparse-view Sinogram Inpainting with Frequency Domain Convolution Enhanced Diffusion Models Jiaze E, Srutarshi Banerjee, Tekin Bicer, Guannan Wang, Bin Ren | arXiv 2024 | Paper/ |
| 2024/08 | - | Iterative CT Reconstruction via Latent Variable Optimization of Shallow Diffusion Models Sho Ozaki, Shizuo Kaji, Toshikazu Imae, Kanabu Nawa, Hideomi Yamashita, Keiichi Nakagawa | arXiv 2024 | Paper/ |
| 2024/08 | - | Four-Dimensional Cone-Beam CT Reconstruction via Diffusion Model and Motion Compensation Xianghong Wang, Zhengwei Ou, Peng Jin, Jiayi Xie, Ze Teng, Lei Xu | TRPMS 2024 | Paper/ |
| 2024/08 | - | Linear diffusion noise boosted deep image prior for unsupervised sparse-view CT reconstruction Jia Wu, Xiaoming Jiang, Lisha Zhong, Wei Zheng, Xinwei Li, Jinzhao Lin, Zhangyong Li | IOP 2024 | Paper/ |
| 2024/07 | DiffRecon | DiffRecon: Diffusion-based CT reconstruction with cross-modal deformable fusion for DR-guided non-coplanar radiotherapy Jiawei Sun, Nannan Cao, Hui Bi, Liugang Gao, Kai Xie, Tao Lin, Jianfeng Sui, Xinye Ni | COMPUT. BIOL. MED. 2024 | Paper/ |
| 2024/07 | SAD | Structure-aware diffusion for low-dose CT imaging Wenchao Du, HuanHuan Cui, LinChao He, Hu Chen, Yi Zhang, Hongyu Yang | IOP 2024 | Paper/ |
| 2024/06 | PFGDM | Prior frequency guided diffusion model for limited angle (LA)-CBCT reconstruction Jiacheng Xie, Hua-Chieh Shao, Yunxiang Li, You Zhang | IOP 2024 | Paper/ |
| 2024/06 | TIFA | Time-reversion Fast-sampling Score-based Model for Limited-angle CT Reconstruction Yanyang Wang, Zirong Li, Weiwen Wu | TMI 2024 | Paper/Code |
| 2024/06 | DiffusionBlend | DiffusionBlend: Learning 3D Image Prior through Position-aware Diffusion Score Blending for 3D Computed Tomography Reconstruction Bowen Song, Jason Hu, Zhaoxu Luo, Jeffrey A. Fessler, Liyue Shen | arXiv 2024 | Paper/ |
| 2024/05 | Blaze3DM | Blaze3DM: Marry Triplane Representation with Diffusion for 3D Medical Inverse Problem Solving Jia He, Bonan Li, Ge Yang, Ziwen Liu | arXiv 2024 | Paper/ |
| 2024/05 | PDHG | Physics-informed Score-based Diffusion Model for Limited-angle Reconstruction of Cardiac Computed Tomography Shuo Han, Yongshun Xu, Dayang Wang, Bahareh Morovati, Li Zhou, Jonathan S. Maltz, Ge Wang, Hengyong Yu | arXiv 2024 | Paper/ |
| 2024/05 | - | Score-based generative model-assisted information compensation for high-quality limited-view reconstruction in photoacoustic tomography Kangjun Guo, Zhiyuan Zheng, Wenhua Zhong, Zilong Li, Guijun Wang,Jiang Li, Yubin Cao,Yiguang Wang,Jiabin Lin, Qiegen Liu, XianLin Song | Photoacoustics 2024 | Paper/ |
| 2024/05 | CDDM | Mitigating Data Consistency Induced Discrepancy in Cascaded Diffusion Models for Sparse-view CT Reconstruction Hanyu Chen, Zhixiu Hao, Lin Guo, Liying Xiao | arXiv 2024 | Paper/ |
| 2024/05 | MSDiff | MSDiff: Multi-Scale Diffusion Model for Ultra-Sparse View CT Reconstruction Pinhuang Tan, Mengxiao Geng, Jingya Lu, Liu Shi, Bin Huang, Qiegen Liu | arXiv 2024 | Paper/ |
| 2024/04 | - | Iterative reconstruction for limited-angle CT using implicit neural representation Jooho Lee, Jongduk Baek | IOP 2024 | Paper/ |
| 2024/04 | DPER | DPER: Diffusion Prior Driven Neural Representation for Limited Angle and Sparse View CT Reconstruction Chenhe Du, Xiyue Lin, Qing Wu, Xuanyu Tian, Ying Su, Zhe Luo, Rui Zheng, Yang Chen, Hongjiang Wei, S. Kevin Zhou, Jingyi Yu, Yuyao Zhang | arXiv 2024 | Paper/ |
| 2024/04 | - | An interactive method based on multi-objective optimization for limited-angle CT reconstruction Chengxiang Wang, Yuanmei Xia, Jiaxi Wang, Kequan Zhao, Wei Peng, Wei Yu | IOP 2024 | Paper/ |
| 2024/04 | - | Fourier diffusion for sparse CT reconstruction Anqi Liu, Grace J. Gang, J. Webster Stayman | SPIE 2024 | Paper/ |
| 2024/03 | DE-CBCT | Dual-Energy Cone-Beam CT Using Two Complementary Limited-Angle Scans with A Projection-Consistent Diffusion Model Junbo Peng, Chih-Wei Chang, Richard L.J. Qiu, Tonghe Wang, Justin Roper, Beth Ghavidel, Xiangyang Tang, Xiaofeng Yang | arXiv 2024 | Paper/ |
| 2024/03 | cDDPM | Using denoising diffusion probabilistic models to enhance quality of limited-view photoacoustic tomography Bruno De Santi, Navchetan Awasthi, Srirang Manohar | SPIE 2024 | Paper/ |
| 2024/03 | - | Image reconstruction based on nonlinear diffusion model for limited-angle computed tomography Xuying Zhao1, Wenjin Jiang, Xinting Zhang, Wenxiu Guo, Yunsong Zhao, Xing Zhao | IOP 2024 | Paper/ |
| 2024/02 | - | Image Domain Ultra-Sparse View CT Artifact Removal Via Conditional Denoising Diffusion Probability Model Feixiang Zhao, Jianchao Zhao, Mingzhe Liu | ICCPR 2024 | Paper/ |
| 2024/02 | WISM | Wavelet-Inspired Multi-channel Score-based Model for Limited-angle CT Reconstruction Jianjia Zhang, Haiyang Mao, Xinran Wang, Yuan Guo, Weiwen Wu | TMI 2024 | Paper/ |
| 2024/01 | SWORD | Stage-by-stage Wavelet Optimization Refinement Diffusion Model for Sparse-View CT Reconstruction Kai Xu, Shiyu Lu, Bin Huang, Weiwen Wu, Qiegen Liu | TMI 2024 | Paper/ |
| 2023/12 | HD-DCDM | HD-DCDM: Hybrid-domain network for limited-angle computed tomography with deconvolution and conditional diffusion model Jianyu Wang, Rongqian Wang1, Lide Cai, Xintong Liu1, Guochang Lin1, Fukai Chen1, Lingyun Qiu | AMMC 2023 | Paper/ |
| 2023/10 | - | Enhancing the Resolution of Micro-CT Images of Rock Samples via Unsupervised Machine Learning based on a Diffusion Model Zhaoyang Ma, Shuyu Sun, Bicheng Yan, Hyung Kwak, Jun Gao | SPEATCE 2023 | Paper/ |
| 2023/08 | - | Generative Modeling in Sinogram Domain for Sparse-View CT Reconstruction Bing Guan, Cailian Yang, Liu Zhang, Shanzhou Niu, Minghui Zhang, Yuhao Wang | TRPMS 2023 | Paper/ |
| 2023/07 | HGU | Fast and Stable Diffusion Inverse Solver with History Gradient Update Linchao He, Hongyu Yan, Mengting Luo, Hongjie Wu, Kunming Luo, Wang Wang, Wenchao Du, Hu Chen, Hongyu Yang, Yi Zhang, Jiancheng Lv | arXiv 2023 | Paper/ |
| 2022/11 | DOLCE | DOLCE: A Model-Based Probabilistic Diffusion Framework for Limited-Angle CT Reconstruction Jiaming Liu, Rushil Anirudh, Jayaraman J. Thiagarajan, Stewart He, K. Aditya Mohan, Ulugbek S. Kamilov, Hyojin Kim | ICCV 2023 | Paper/Code |
| 2022/11 | DiffusionMBIR | Solving 3D Inverse Problems using Pre-trained 2D Diffusion Models Hyungjin Chung, Dohoon Ryu, Michael T. McCann, Marc L. Klasky, Jong Chul Ye | CVPR 2023 | Paper/Code |
| 2022/01 | MCG | Improving diffusion models for inverse problems using manifold constraints Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, Jong Chul Ye | NeurIPS 2022 | Paper/Code |
2.1.3 Multi-modal
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/10 | PASTA | PASTA: Pathology-Aware MRI to PET CroSs-modal TrAnslation with Diffusion Models Yitong Li, Igor Yakushev, Dennis M. Hedderich, Christian Wachinger | MICCAI 2024 | Paper/Code |
| 2024/07 | Joint diffusion | Joint diffusion: mutual consistency-driven diffusion model for PET-MRI co-reconstruction Taofeng Xie, Zhuo-Xu Cui, Chen Lu, Huayu Wang, Congcong Liu, Yuanzhi Zhang, Xuemei Wang, Yanjie Zhu, Guoqing Chen, Dong Liang | IOP 2024 | Paper/ |
| 2024/05 | FICD | Functional Imaging Constrained Diffusion for Brain PET Synthesis from Structural MRI Minhui Yu, Mengqi Wu, Ling Yue, Andrea Bozoki, Mingxia Liu | arXiv 2024 | Paper/ |
| 2023/06 | SynDiff | Unsupervised Medical Image Translation With Adversarial Diffusion Models Muzaffer Özbey; Onat Dalmaz; Salman U. H. Dar; Hasan A. Bedel; Şaban Özturk; Alper Güngör | TMI 2023 | Paper/Code |
| 2023/04 | FGDM | Zero-shot Medical Image Translation via Frequency-Guided Diffusion Models Yunxiang Li, Hua-Chieh Shao, Xiao Liang, Liyuan Chen, Ruiqi Li, Steve Jiang, Jing Wang, You Zhang | TMI 2023 | Paper/ |
| 2022/09 | - | Conversion Between CT and MRI Images Using Diffusion and Score-Matching Models Qing Lyu, Ge Wang | arXiv 2022 | Paper/ |
| 2022/07 | UMM-CSGM | A Novel Unified Conditional Score-based Generative Framework for Multi-modal Medical Image Completion Xiangxi Meng, Yuning Gu, Yongsheng Pan, Nizhuan Wang, Peng Xue, Mengkang Lu, Xuming He, Yiqiang Zhan, Dinggang Shen | arXiv 2022 | Paper/ |
2.1.4 Other Modalities
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/10 | MCAD | MCAD: Multi-modal Conditioned Adversarial Diffusion Model for High-Quality PET Image Reconstruction Jiaqi Cui, Xinyi Zeng, Pinxian Zeng, Bo Liu, Xi Wu, Jiliu Zhou, Yan Wang | MICCAI 2024 | Paper/ |
| 2024/08 | PET-CM | Full-dose whole-body PET synthesis from low-dose PET using high-efficiency denoising diffusion probabilistic model: PET consistency model Shaoyan Pan, Elham Abouei, Junbo Peng, Joshua Qian, Jacob F Wynne, Tonghe Wang, Chih-Wei Chang, Justin Roper, Jonathon A Nye, Hui Mao, Xiaofeng Yang | Medical Physics 2024 | Paper/Code |
| 2024/04 | Entropy-SDE | Equipping Diffusion Models with Differentiable Spatial Entropy for Low-Light Image Enhancement Wenyi Lian, Wenjing Lian, Ziwei Luo | CVPRW 2024 | Paper/Code |
| 2024/02 | - | Dehazing Ultrasound using Diffusion Models Tristan S.W. Stevens, Faik C. Meral, Jason Yu, Iason Z. Apostolakis, Jean-Luc Robert, Ruud J.G. Van Sloun | TMI 2024 | Paper/ |
| 2023/10 | LLCaps | LLCaps: Learning to Illuminate Low-Light Capsule Endoscopy with Curved Wavelet Attention and Reverse Diffusion Tristan S.W. Stevens, Faik C. Meral, Jason Yu, Iason Z. Apostolakis, Jean-Luc Robert, Ruud J.G. Van Sloun | MICCAI 2023 | Paper/Code |
| 2023/10 | CPDM | Content-Preserving Diffusion Model for Unsupervised AS-OCT Image Despeckling Sanqian Li, Risa Higashita, Huazhu Fu, Heng Li, Jingxuan Niu, Jiang Liu | MICCAI 2023 | Paper/ |
| 2023/10 | PET-Diffusion | PET-Diffusion: Unsupervised PET Enhancement Based on the Latent Diffusion Model Caiwen Jiang, Yongsheng Pan, Mianxin Liu, Lei Ma, Xiao Zhang, Jiameng Liu, Xiaosong Xiong, Dinggang Shen | MICCAI 2023 | Paper/Code |
| 2023/10 | - | PET image denoising based on denoising diffusion probabilistic model Kuang Gong, Keith Johnson, Georges El Fakhri, Quanzheng Li, Tinsu Pan | EJNMMI 2024 | Paper/ |
| 2023/10 | PET-Reconstruction | Contrastive Diffusion Model with Auxiliary Guidance for Coarse-to-Fine PET Reconstruction Zeyu Han, Yuhan Wang, Luping Zhou, Peng Wang, Binyu Yan, Jiliu Zhou, Yan Wang, Dinggang Shen | MICCAI 2023 | Paper/Code |
| 2022/09 | PET-DDM | PET image denoising based on denoising diffusion probabilistic models Kuang Gong, Keith A. Johnson, Georges El Fakhri, Quanzheng Li, Tinsu Pan | Eur. J. Nucl. Med. Mol. Imaging 2023 | Paper/ |
| 2022/04 | OCT-DDPM | Unsupervised denoising of retinal OCT with diffusion probabilistic model Dewei Hu, Yuankai K. Tao, Ipek Oguz | SPIE 2022 | Paper/Code |
| 2022/01 | DenoOCT-DDPM | Unsupervised Denoising of Retinal OCT with Diffusion Probabilistic Model Dewei Hu, Yuankai K. Tao, Ipek Oguz | SPIE 2022 | Paper/Code |
2.2 Diffusion Models In Remote Sensing For Low-level Vision Tasks
2.2.1 Visible-light Remote Sensing Image
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/12 | EVADM | Effective variance attention-enhanced diffusion model for crop field aerial image super resolution Xiangyu Lu, Jianlin Zhang, Rui Yang , Qina Yang , Mengyuan Chen , Hongxing Xu, Pinjun Wan, Jiawen Guo , Fei Liu | ISPRS 2024 | Paper/Code |
| 2024/09 | FastDiffSR | A Conditional Diffusion Model With Fast Sampling Strategy for Remote Sensing Image Super-Resolution Fanen Meng; Yijun Chen; Haoyu Jing; Laifu Zhang; Yiming Yan; Yingchao Ren | TGRS 2024 | Paper/Code |
| 2024/09 | - | Clouds and Haze Co-Removal Based on Weight-Tuned Overlap Refinement Diffusion Model for Remote Sensing Images Jingxuan Zhang; Libao Zhang | ICIP 2024 | Paper/ |
| 2024/07 | RSSRDiff | RSSRDiff: An Effective Diffusion Probability Model with Attention for Single Remote Sensing Image Super-Resolution Tian Wei, Hanyi Zhang, Jin Xu, Jing Zhao, Fei Shen | ICIC 2024 | Paper/ |
| 2024/06 | - | Diffusion Enhancement for Cloud Removal in Ultra-Resolution Remote Sensing Imagery Jialu Sui; Yiyang Ma; Wenhan Yang; Xiaokang Zhang; Man-On Pun; Jiaying Liu | TGRS 2024 | Paper/Code |
| 2024/06 | CLDiff | CLDiff: Weakly Supervised Cloud Detection With Denoising Diffusion Probabilistic Models | TGRS 2024 | Paper/Code |
| 2024/05 | SGDM | Semantic Guided Large Scale Factor Remote Sensing Image Super-resolution with Generative Diffusion Prior Ce Wang, Wanjie Sun | arXiv 2024 | Paper/Code |
| 2024/05 | ShipinSight | Ship in Sight: Diffusion Models for Ship-Image Super Resolution Luigi Sigillo, Riccardo Fosco Gramaccioni, Alessandro Nicolosi, Danilo Comminiello | WCCI 2024 | Paper/Code |
| 2024/05 | RSHazeDiff | RSHazeDiff: A Unified Fourier-aware Diffusion Model for Remote Sensing Image Dehazing Jiamei Xiong, Xuefeng Yan, Yongzhen Wang, Wei Zhao, Xiao-Ping Zhang, Mingqiang Wei | arXiv 2024 | Paper/Code |
| 2024/03 | ASDDPM | Adaptive Semantic-Enhanced Denoising Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution Jialu Sui, Xianping Ma, Xiaokang Zhang, Man-On Pun | arXiv 2024 | Paper/Code |
| 2024/03 | IDF-CR | IDF-CR: Iterative Diffusion Process for Divide-and-Conquer Cloud Removal in Remote-Sensing Images Meilin Wang; Yexing Song; Pengxu Wei; Xiaoyu Xian; Yukai Shi; Liang Lin | TGRS 2024 | Paper/Code |
| 2024/03 | DiffALS | Denoising Diffusion Probabilistic Model with Adversarial Learning for Remote Sensing Super-Resolution Jialu Sui,Qianqian Wu， Man-On Pun | Remote Sensing 2024 | Paper/ |
| 2024/02 | ADND-Net | Diffusion Models Based Null-Space Learning for Remote Sensing Image Dehazing Yufeng Huang; Zhiyu Lin; Shuai Xiong; Tongtong Sun | LGRS 2024 | Paper |
| 2024/02 | DiffCR | DiffCR: A Fast Conditional Diffusion Framework for Cloud Removal From Optical Satellite Images Xuechao Zou; Kai Li; Junliang Xing; Yu Zhang; Shiying Wang; Lei Jin | TGRS 2024 | Paper/Code |
| 2024/02 | TCDM | TCDM: Effective Large-Factor Image Super-Resolution via Texture Consistency Diffusion Yan Zhang; Hanqi Liu; Zhenghao Li; Xinbo Gao; Guangyao Shi; Jianan Jiang | TGRS 2024 | Paper/ |
| 2023/11 | LWTDM | Efficient Remote Sensing Image Super-Resolution via Lightweight Diffusion Models Tai An; Bin Xue; Chunlei Huo; Shiming Xiang | LGRS 2023 | Paper/Code |
| 2023/10 | EDiffSR | EDiffSR: An Efficient Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution Yi Xiao, Qiangqiang Yuan, Kui Jiang, Jiang He, Xianyu Jin, Liangpei Zhang | TGRS 2024 | Paper/Code |
| 2023/08 | DDSR | Dual-Diffusion: Dual Conditional Denoising Diffusion Probabilistic Models for Blind Super-Resolution Reconstruction in RSIs Mengze Xu, Jie Ma, Yuanyuan Zhu | LGRS 2024 | Paper/Code |
| 2023/07 | EHC-DMSR | Enhancing Remote Sensing Image Super-Resolution with Efficient Hybrid Conditional Diffusion Model Lintao Han,Yuchen Zhao, Hengyi Lv, Yisa Zhang, Hailong Liu, Guoling Bi, Qing Han | Remote Sensing 2023 | Paper/ |
| 2023/09 | RSDiff | RSDiff: Remote Sensing Image Generation from Text Using Diffusion Model Ahmad Sebaq, Mohamed ElHelw | arXiv 2023 | Paper/ |
| 2023/08 | ARDD-Net | Remote Sensing Image Dehazing Using Adaptive Region-Based Diffusion Models Y Huang, S Xiong | LGRS 2023 | Paper/ |
| 2023/02 | TESR | TESR: Two-Stage Approach for Enhancement and Super-Resolution of Remote Sensing Images Anas M. Ali, Bilel Benjdira, Anis Koubaa, Wadii Boulila, Walid El-Shafai | Remote Sensing 2023 | Paper/Code |
| 2022/09 | PSSR | Diffusion Model with Detail Complement for Super-Resolution of Remote Sensing Liu, Jinzhe and Yuan, Zhiqiang and Pan, Zhaoying and Fu, Yiqun and Liu, Li and Lu, Bin | Remote Sensing 2022 | Paper/ |
2.2.2 HSI
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/11 | STFDiff | STFDiff: Remote sensing image spatiotemporal fusion with diffusion models He Huang, Wei He, Hongyan Zhang, Yu Xia, Liangpei Zhang | Inf. Fusion 2024 | Paper/Code |
| 2024/11 | MTLSC-Diff | MTLSC-Diff: Multitask learning with diffusion models for hyperspectral image super-resolution and classification Jiahui Qu, Liusheng Xiao, Wenqian Dong, Yunsong Li | Knowledge-Based Systems 2024 | Paper |
| 2024/09 | Diff-Unmix | Unmixing Diffusion for Self-Supervised Hyperspectral Image Denoising Haijin Zeng; Jiezhang Cao; Kai Zhang; Yongyong Chen; Hiep Luong; Wilfried Philips | CVPR 2024 | Paper/Code |
| 2024/08 | SCDM | Spectral-Cascaded Diffusion Model for Remote Sensing Image Spectral Super-Resolution Bowen Chen; Liqin Liu; Chenyang Liu; Zhengxia Zou; Zhenwei Shi | TGRS 2024 | Paper/Code |
| 2024/08 | SDP | A Spectral Diffusion Prior for Unsupervised Hyperspectral Image Super-Resolution Jianjun Liu; Zebin Wu; Liang Xiao | TGRS 2024 | Paper/Code |
| 2024/07 | PLRDiff | Unsupervised hyperspectral pansharpening via low-rank diffusion model Xiangyu Rui, Xiangyong Cao, Li Pang, Zeyu Zhu, Zongsheng Yue, Deyu Meng | Information Fusion 2024 | Paper/Code |
| 2024/05 | ISPDiff | ISPDiff: Interpretable Scale-Propelled Diffusion Model for Hyperspectral Image Super-Resolution Wenqian Dong; Sen Liu; Song Xiao; Jiahui Qu; Yunsong Li | TGRS 2024 | Paper/Code |
| 2024/04 | DMSANet | A Diffusion Model-Assisted Multiscale Spectral Attention Network for Hyperspectral Image Super-Resolution Kaiqi He; Yiheng Cai; Shengjun Peng; Meiling Tan | JSTARS 2024 | Paper |
| 2024/03 | DMGASR | Enhancing Hyperspectral Images via Diffusion Model and Group-Autoencoder Super-resolution Network Zhaoyang Wang, Dongyang Li, Mingyang Zhang†, Hao Luo, Maoguo Gong | AAAI 2024 | Paper/Code |
| 2024/02 | HIR-Diff | HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models Li Pang, Xiangyu Rui, Long Cui, Hongzhong Wang, Deyu Meng, Xiangyong Cao | CVPR 2024 | Paper/Code |
| 2024/02 | CFMDM | CFMDM: Coarse-to-Fine Meta-Diffusion Model for Scale-Arbitrary Hyperspectral Super-Resolution Jizhou Cui; Wenqian Dong; Jiahui Qu; Xiaoyang Wu; Song Xiao; Yunsong Li | LGRS 2024 | Paper/ |
| 2024/01 | SatDiffMoE | SatDiffMoE: A Mixture of Estimation Method for Satellite Image Super-resolution with Latent Diffusion Models Zhaoxu Luo, Bowen Song, Liyue Shen | arXiv 2024 | Paper/ |
| 2023/11 | TDiffDe | TDiffDe: A Truncated Diffusion Model for Remote Sensing Hyperspectral Image Denoising Jiang He, Yajie Li, Jie L, Qiangqiang Yuan | arXiv 2024 | Paper/ |
| 2023/07 | DDPM-Fus | Hyperspectral and Multispectral Image Fusion Using the Conditional Denoising Diffusion Probabilistic Model Shuaikai Shi, Lijun Zhang, Jie Chen | arXiv 2023 | Paper/Code |
| 2023/07 | - | A Noise-Model-Free Hyperspectral Image Denoising Method Based on Diffusion Model Deng, Keli and Jiang, Zhongshun and Qian, Qipeng and Qiu, Yi and Qian, Yuntao | IGASS 2023 | Paper/ |
| 2023/07 | R2H-CCD | R2H-CCD: Hyperspectral Imagery Generation from RGB Images Based on Conditional Cascade Diffusion Probabilistic Models Zhang, Lei and Luo, Xiaoyan and Li, Sen and Shi, Xiaofeng | IGASS 2023 | Paper/ |
| 2023/03 | DDS2M | DDS2M: Self-Supervised Denoising Diffusion Spatio-Spectral Model for Hyperspectral Image Restoration Yuchun Miao, Lefei Zhang, Liangpei Zhang, Dacheng Tao | ICCV 2023 | Paper/Code |
| 2023/01 | HSR-Diff | HSR-Diff:Hyperspectral Image Super-Resolution via Conditional Diffusion Models Chanyue Wu, Dong Wang, Hanyu Mao, Ying Li | ICCV 2023 | Paper/ |
2.2.3 SAR
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/01 | Ship-Go | Ship-Go: SAR Ship Images Inpainting via instance-to-image Generative Diffusion Models Xin Zhang, Yang Li, Feng Li , Hangzhi Jiang, Yanhua Wang , Liang Zhang , Le Zheng, Zegang Ding | ISPRS 2024 | Paper/Code |
| 2024/09 | R-DDPM | SAR Despeckling Via Regional Denoising Diffusion Probabilistic Model Xuran Hu; Ziqiang Xu; Zhihan Chen; Zhenpeng Feng; Mingzhe Zhu; Ljubiša Stanković | IGASS 2024 | Paper/ |
| 2024/06 | - | Despeckling SAR Images With Log-Yeo–Johnson Transformation and Conditional Diffusion Models Yaobin Ma; Peng Ke; Hossein Aghababaei; Ling Chang; Jingbo Wei | TGRS 2024 | Paper/ |
| 2023/07 | - | Unsupervised SAR Despeckling Based on Diffusion Model Xiao, Siyao and Huang, Libing and Zhang, Shunsheng | IGASS 2023 | Paper/ |
| 2023/08 | - | Diffusion Models for Interferometric Satellite Aperture Radar Alexandre Tuel, Thomas Kerdreux, Claudia Hulbert, Bertrand Rouet-Leduc | arXiv 2023 | Paper/Code |
| 2023/06 | - | SAR Despeckling using a Denoising Diffusion Probabilistic Model Malsha V. Perera, Nithin Gopalakrishnan Nair, Wele Gedara Chaminda Bandara, Vishal M. Patel | LGRS 2023 | Paper/Code |
2.2.4 Multi-modal
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/10 | - | Binary Diffusion Method for Image Fusion with Despeckling Chenlin Zhang; Yajun Chang; Yuhang Wu; Zelong Wang | AIPMV 2024 | Paper/ |
| 2024/09 | - | SAR to Optical Image Translation with Color Supervised Diffusion Model Xinyu Bai; Feng Xu | IGARSS 2024 | Paper/ |
| 2024/08 | - | Learning SAR-to-Optical Image Translation via Diffusion Models With Color Memory Zhe Guo; Jiayi Liu; Qinglin Cai; Zhibo Zhang; Shaohui Mei | JSTARS 2024 | Paper/ |
| 2024/07 | DDSR | DDSR: Degradation-Aware Diffusion Model for Spectral Reconstruction from RGB Images Yunlai Chen, Xiaoyan Zhang | Remote Sensing 2024 | Paper/ |
| 2024/04 | - | Variational Diffusion Method for Remote Sensing Image Fusion Chenlin Zhang; Jialing Han; Jubo Zhu; Zelong Wang | LGRS 2024 | Paper/ |
| 2024/01 | - | A brain-inspired approach for SAR-to-optical image translation based on diffusion models Hao Shi, Zihan Cui, Liang Chen, ingfei He, Jingyi Yang | FRONT NEUROSCI-SWITZ 2024 | Paper/ |
| 2023/11 | - | Conditional Diffusion for SAR to Optical Image Translation Xinyu Bai; Xinyang Pu; Feng Xu | LGRS 2024 | Paper/ |
| 2023/10 | DCDMF | Hyperspectral and Panchromatic Images Fusion Based on the Dual Conditional Diffusion Models Shuangliang Li; Siwei Li; Lihao Zhang | TGRS 2024 | Paper/Code |
| 2023/07 | - | Improved Flood Insights: Diffusion-Based SAR to EO Image Translation Minseok Seo, Youngtack Oh, Doyi Kim, Dongmin Kang, Yeji Choi | arXiv 2023 | Paper/ |
| 2023/04 | - | Cloud Removal in Remote Sensing Using Sequential-Based Diffusion Models Zhao, Xiaohu and Jia, Kebin | Remote Sensing 2023 | Paper/ |
| 2023/04 | DDRF | DDRF: Denoising Diffusion Model for Remote Sensing Image Fusion ZiHan Cao, ShiQi Cao, Xiao Wu, JunMing Hou, Ran Ran, Liang-Jian Deng | arXiv 2023 | Paper/ |
| 2023/03 | DDPM-CR | Denoising Diffusion Probabilistic Feature-Based Network for Cloud Removal in Sentinel-2 Imagery Jing, Ran and Duan, Fuzhou and Lu, Fengxian and Zhang, Miao and Zhao, Wenji | Remote Sensing 2023 | Paper/ |
2.3 Varied Low-level Vision Tasks In Video Through Diffusion Models
2.3.1 Video Frame Prediction and Interpolation
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/09 | ExtDM | ExtDM: Distribution Extrapolation Diffusion Model for Video Prediction Zhicheng Zhang, Junyao Hu, Wentao Cheng, Danda Paudel, Jufeng Yang | CVPR 2024 | Paper/ |
| 2024/08 | EasyControl | EasyControl: Transfer ControlNet to Video Diffusion for Controllable Generation and Interpolation Cong Wang, Jiaxi Gu, Panwen Hu, Haoyu Zhao, Yuanfan Guo, Jianhua Han, Hang Xu, Xiaodan Liang | arXiv 2024 | Paper/ |
| 2024/04 | MADiff | Motion-aware Latent Diffusion Models for Video Frame Interpolation Zhilin Huang, Yijie Yu, Ling Yang, Chujun Qin, Bing Zheng, Xiawu Zheng, Zikun Zhou, Yaowei Wang, Wenming Yang | arXiv 2024 | Paper/ |
| 2024/04 | VIDIM | Video Interpolation with Diffusion Models Siddhant Jain, Daniel Watson, Eric Tabellion, Aleksander Hołyński, Ben Poole, Janne Kontkanen | CVPR 2023 | Paper/ |
| 2024/03 | STDiff | STDiff: Spatio-Temporal Diffusion for Continuous Stochastic Video Prediction Xi Ye, Guillaume-Alexandre Bilodeau | AAAI 2024 | Paper/Code |
| 2024/01 | AID | AID: Adapting Image2Video Diffusion Models for Instruction-guided Video Prediction Zhen Xing, Qi Dai, Zejia Weng, Zuxuan Wu, Yu-Gang Jiang | arXiv 2024 | Paper/ |
| 2023/05 | Seer | Seer: Language Instructed Video Prediction with Latent Diffusion Models Xianfan Gu, Chuan Wen, Weirui Ye, Jiaming Song, Yang Gao | arXiv 2023 | Paper/ |
| 2023/10 | SEINE | SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction Xinyuan Chen, Yaohui Wang, Lingjun Zhang, Shaobin Zhuang, Xin Ma, Jiashuo Yu, Yali Wang, Dahua Lin, Yu Qiao, Ziwei Liu | ICLR 2024 | Paper/Project |
| 2023/03 | LDMVFI | LDMVFI: Video Frame Interpolation with Latent Diffusion Models Duolikun Danier, Fan Zhang, David Bull | arXiv 2023 | Paper/Code |
| 2022/06 | RaMViD | Diffusion Models for Video Prediction and Infilling Tobias Höppe, Arash Mehrjou, Stefan Bauer, Didrik Nielsen, Andrea Dittadi | TMLR 2022 | Paper/ |
| 2022/05 | MCVD | MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation Vikram Voleti, Alexia Jolicoeur-Martineau, Christopher Pal | NeurIPS 2022 | Paper/Code |
| 2022/03 | RVD | Diffusion Probabilistic Modeling for Video Generation Ruihan Yang, Prakhar Srivastava, Stephan Mandt | Entropy 2023 | Paper/Code |
2.3.2 Super Resolution For Video Generation
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/06 | HPDM | Hierarchical Patch Diffusion Models for High-Resolution Video Generation Ivan Skorokhodov, Willi Menapace, Aliaksandr Siarohin, Sergey Tulyakov | CVPR 2024 | Paper/Project |
| 2024/03 | SATeCo | Learning Spatial Adaptation and Temporal Coherence in Diffusion Models for Video Super-Resolution Zhikai Chen, Fuchen Long, Zhaofan Qiu, Ting Yao, Wengang Zhou, Jiebo Luo, Tao Mei | CVPR 2024 | Paper/ |
| 2024/01 | - | Inflation with Diffusion: Efficient Temporal Adaptation for Text-to-Video Super-Resolution Xin Yuan, Jinoo Baek, Keyang Xu, Omer Tov, Hongliang Fei | WACV 2024 | Paper/ |
| 2023/12 | Upscale-A-Video | Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution Shangchen Zhou, Peiqing Yang, Jianyi Wang, Yihang Luo, Chen Change Loy | CVPR 2024 | Paper/Code |
| 2023/12 | MGLD-VSR | Motion-Guided Latent Diffusion for Temporally Consistent Real-world Video Super-resolution Xi Yang, Chenhang He, Jianqi Ma, Lei Zhang | ECCV 2024 | Paper/Code |
| 2023/11 | StableVSR | Enhancing Perceptual Quality in Video Super-Resolution through Temporally-Consistent Detail Synthesis using Diffusion Models Claudio Rota, Marco Buzzelli, Joost van de Weijer | ECCV 2024 | Paper/Code |
| 2023/09 | LAVIE | LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, Yuwei Guo, Tianxing Wu, Chenyang Si, Yuming Jiang, Cunjian Chen, Chen Change Loy, Bo Dai, Dahua Lin, Yu Qiao, Ziwei Liu | arXiv 2023 | Paper/Code |
| 2023/05 | PYoCo | Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming-Yu Liu, Yogesh Balaji | ICCV 2023 | Paper/Demo |
| 2023/05 | VideoFusion | VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation Zhengxiong Luo, Dayou Chen, Yingya Zhang, Yan Huang, Liang Wang, Yujun Shen, Deli Zhao, Jingren Zhou, Tieniu Tan | CVPR 2023 | Paper/ |
| 2023/04 | - | Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, Karsten Kreis | CVPR 2023 | Paper/Demo |
2.3.3 Video Restoration
| Release | Method | Title | Pub. | Link |
|---|---|---|---|---|
| 2024/08 | VD-Diff | Rethinking Video Deblurring with Wavelet-Aware Dynamic Transformer and Diffusion Model Chen Rao, Guangyuan Li, Zehua Lan, Jiakai Sun, Junsheng Luan, Wei Xing, Lei Zhao, Huaizhong Lin, Jianfeng Dong, Dalong Zhang | ECCV 2024 | Paper/Code |
| 2024/08 | - | Nonlinear Reaction-Diffusion Based Video Restoration Technique for Noise Mixtures Tudor Barbu, Costică Moroşanu | ICIEA 2024 | Paper/ |
| 2024/07 | DiffIR2VR-Zero | DiffIR2VR-Zero: Zero-Shot Video Restoration with Diffusion-based Image Restoration Models Chang-Han Yeh, Chin-Yang Lin, Zhixiang Wang, Chi-Wei Hsiao, Ting-Hsuan Chen, Yu-Lun Liu | arXiv 2024 | Paper/Code |
| 2024/07 | - | Zero-shot Video Restoration and Enhancement Using Pre-Trained Image Diffusion Model Cong Cao, Huanjing Yue, Xin Liu, Jingyu Yang | arXiv 2024 | Paper/ |
| 2024/03 | DiffTTA | Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal Yijun Yang, Hongtao Wu, Angelica I. Aviles-Rivero, Yulun Zhang, Jing Qin, Lei Zhu | CVPR 2024 | Paper/Code |
| 2023/12 | AVID | AVID: Any-Length Video Inpainting with Diffusion Model Zhixing Zhang, Bichen Wu, Xiaoyan Wang, Yaqiao Luo, Luxin Zhang, Yinan Zhao, Peter Vajda, Dimitris Metaxas, Licheng Yu | CVPR 2024 | Paper/Code |
| 2023/11 | FLAIR | FLAIR: A Conditional Diffusion Framework with Applications to Face Video Restoration Zihao Zou, Jiaming Liu, Shirin Shoushtari, Yubo Wang, Weijie Gan, Ulugbek S. Kamilov | arXiv 2024 | Paper/ |
Related Surveys Recommended
Diffusion Models in Low-Level Vision: A Survey
arXiv 2024. [Paper]
Jun. 2024
Taming Diffusion Models for Image Restoration: A Review
arXiv 2024. [Paper]
Sept. 2024
Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectives
arXiv 2024. [Paper]
Apr. 2024
Diffusion Models, Image Super-Resolution And Everything: A Survey
arXiv 2024. [Paper]
Jan. 2024
State of the Art on Diffusion Models for Visual Computing
arXiv 2023. [Paper]
Oct. 2023
Diffusion Models for Image Restoration and Enhancement -- A Comprehensive Survey.
arXiv 2023. [Paper]
Aug. 2023
Survey on Diverse Image Inpainting using Diffusion Models
PCEMS 2023. [Paper]
Jun. 2023
Diffusion Models for Medical Image Analysis: A Comprehensive Survey
Medical Image Analysis 2023. [Paper]
Nov. 2022
Diffusion Models in Vision: A Survey
TPAMI 2023. [Paper]
Sep. 2022
Diffusion Models: A Comprehensive Survey of Methods and Applications
ACM Computing Surveys 2023. [Paper]
Sep. 2022
Large-scale datasets for model pre-training
Here, we provide a more comprehensive overview of the commonly used Large-scale datasets for model pre-training in low-level vision tasks.
- ImageNet
ImageNet is a large-scale dataset with over 14 million natural images spanning over 21k classes, termed ImageNet21K. ImageNet1k, serving as a subset of ImageNet21K, has 1k classes with about 1k images per class, often resized to practical dimensions of 224 × 224 or 256 × 256. - CelebA
CelebA has 200k facial images, each annotated with 40 attributes, featuring 10k celebrities. CelebA-HQ is a subset having 30k high-resolution facial images with a resolution of 1024 × 1024. Enhanced with detailed annotations, CelebAMask-HQ is derived from pixel-wise facial component labeling (face parsing). - LSUN
LSUN includes 10 scene categories and 20 object categories, each having about 1 million labeled images, typically resized with a shorter edge of 256 pixels and compressed to JPEG image quality of 75. - AFHQ
AFHQ comprises around 5,000 high-quality animal face images with three categories: cat, dog, and wildlife, each with a resolution of 512 × 512. Current approaches frequently opt to train diffusion models on specific categories, such as cats. - FFHQ
FFHQ comprises 70k high-resolution facial images with diverse distributions. Existing methods based on pre-trained DMs undergo training on FFHQ and evaluation on CelebA-HQ to showcase their generalizability.
Datasets for low-level vision tasks
Due to space limitations, we provide a summary of commonly used datasets for several classical natural low-level vision tasks here, including their scales, sources, modalities, and remarks. Clicking on the dataset will redirect you to its download link.
| Tasks | Datasets | Scales | Sources | Modalities | Remarks |
|---|---|---|---|---|---|
| SR | BSD500 | 500 | TPAMI 2010 | Syn | A synthetic benchmark that is initially designed for object contour detection. |
| SR | Set14 | 14 | TPAMI 2015 | Syn | Commonly utilized for testing performance of super-resolution algorithms. |
| SR | Manga109 | 109 | MTAP 2015 | Syn | Compiled mainly for academic research on Japanese manga media processing. |
| SR | General100 | 100 | ECCV 2016 | Syn | Synthesized images in uncompressed BMP format covering various scales. |
| SR | DIV2K | 900/100 | NTIRE 2018 | Syn | A commonly-used dataset with diverse scenarios and realistic degradations. |
| SR | Flickr1024 | 1024 | ICCVW 2019 | Syn | A large-scale stereo image dataset with high-quality pairs and diverse scenarios. |
| SR | Urban100 | 100 | CVPR 2019 | Syn | Sourced from urban environments: city streets, buildings, and urban landscapes. |
| SR | DRealSR | 31970 | ECCV 2020 | Real | Benchmarks captured by DSLR cameras, circumventing simulated degradation. |
| Deblur | GoPro | 2103/1111 | CVPR 2017 | Syn | Acquired by high-speed cameras for video quality assessment and restoration. |
| Deblur | HIDE | 8422 | ICCV 2019 | Syn | Cover long-distance and short-distance scenarios degraded by motion blur. |
| Deblur | REDS | 270/30 | NTIRE 2019 | Real | Contain 300 video sequences with dynamic duration and varied resolutions. |
| Deblur | BSD | 80/20 | ECCV 2020 | Real | Comprise more scenes and use the proposed beam-splitter acquisition system. |
| Deblur | RealBlur | 3758/980 | ECCV 2020 | Real | Cover common instances of motion blur, captured in raw and JPEG formats. |
| Dehaze | I-Haze | 35 | NTIRE 2018 | Real | Indoor dataset with real haze for objective image dehazing and evaluation. |
| Dehaze | O-Haze | 45 | NTIRE 2018 | Real | Outdoor dataset with real haze for objective image dehazing and evaluation. |
| Dehaze | Dense-Haze | 33 | ICIP 2019 | Real | Real-world dataset with dense haze for robust single image dehazing methods. |
| Dehaze | RESIDE | 13000/990 | TIP 2019 | Syn+Real | Divided into five subsets to highlight diverse sources and heterogeneous contents. |
| Dehaze | NH-Haze | 55 | CVRPW 2020 | Real | The first non-homogeneous dehazing dataset with realistic haze distribution. |
| Dehaze | Haze-4K | 4000 | MM 2021 | Syn | A large-scale synthetic dataset for image dehazing with varing distributions. |
| LLIE | MIT-Fivek | 4500/500 | CVPR 2011 | Syn | A curated dataset of RAW photos adjusted by skilled retouchers for visual appeal. |
| LLIE | LOLv1 | 485/15 | BMVC 2018 | Real | The first dataset with image pairs from real scenarios for low-light enhancement. |
| LLIE | SID | 5094 | CVPR 2018 | Real | A dataset of raw short-exposure images with their long-exposure reference images. |
| LLIE | SICE | 589 | TIP 2018 | Syn | A large-scale multi-exposure image dataset with complex illumination conditions. |
| LLIE | ExDark | 7363 | CVIU 2019 | Real | Collected in low-light scenarios with 12 classes and instance-level annotations. |
| LLIE | LOLv2-Real | 689/100 | TIP 2021 | Real | A three-step shooting strategy is used to eliminate intra-pair image misalignments. |
| LLIE | LOLv2-Syn | 900/100 | TIP 2021 | Syn | Synthetic dark images mimic real low-light photography via histogram analysis. |
| LLIE | SDSD-Indoor | 62/6 | ICCV 2021 | Real | Indoor dataset collected from dynamic scenes under varying lighting conditions. |
| LLIE | SDSD-Outdoor | 116/10 | ICCV 2021 | Real | Outdoor dataset collected from dynamic scenes under varying lighting conditions. |
| Derain | RaindropClarity | 13368/1818 | ECCV 2024 | Real | A large-scale real-world dataset of paired/triplet images with day and night raindrops and dual-focused images. |
| Derain | Rain100H | 1800/100 | CVPR 2017 | Syn | Comprise synthetic datasets with five types of rain streaks for rain removal. |
| Derain | RainDrop | 861/239 | CVPR 2018 | Syn | Image pairs with raindrop degradation, captured using the setup of dual glasses. |
| Derain | SPA-Data | 638492/1000 | CVPR 2019 | Real | Design a semi-automatic method to generate clean images from real rain streaks. |
| Derain | MPID | 3961/419 | CVPR 2019 | Syn+Real | A large-scale benchmark that focuses on driving and surveillance scenarios. |
| Derain | RainCityscapes | 9432/1188 | CVPR 2019 | Syn | A famous rain removal dataset with paired depth maps for outdoor scenarios. |
| Derain | RainDS | 3450/900 | CVPR 2021 | Syn+Real | A hybrid dataset with both real and synthesized data under diverse scenarios. |
| Derain | RainDirection | 2920/430 | ICCV 2021 | Syn | A large-scale synthetic rainy dataset with directional labels in the training phase. |
| Derain | GT-RAIN | 28217/2100 | ECCV 2022 | Real | The first paired derain dataset with real data by controlling non-rain variations. |
| Desnow | Snow100k | 100000 | TIP 2018 | Syn+Real | A large-scale dataset with over 1k real-world images degraded by heavy snow. |
| Desnow | SRRS | 16000 | ECCV 2020 | Syn+Real | A hybrid snow dataset with 15k synthesized images and 1k real-world images. |
| Desnow | CSD | 10000 | ICCV 2021 | Syn | A large-scale desnowing dataset to comprehensively simulate snow scenarios. |
Evaluation metrics
Due to space limitations, we only introduced the evaluation metrics involved in the comparative experiments in the survey. Here, we provide a more comprehensive overview of the commonly used metrics in low-level vision tasks.
Distortion-based metrics
- PSNR (Peak Signal to Noise Ratio)
PSNR quantifies the pixel-wise disparity between a corrupted image and its clean image by computing their mean squared error. - SSIM (Structural Similarity)
SSIM aims to accommodate human visual perception, assesses the likeness between distorted and clean images across three aspects, including contrast, brightness, and structure.
Inception-based metrics
- LPIPS (Learned Perceptual Image Patch Similarity)
LPIPS is a learning-based metric that leverages the pre-trained AlexNet as a feature extractor and adjusts the linear layer to emulate human perception. - FID (Fréchet inception distance)
FID assesses the fidelity and diversity of generated images by modeling the feature-level multivariate Gaussian distribution of the extracted features, by computing the Fréchet distance of their reference images. - KID (Kernel Inception Distance)
KID is similar to FID, which also leverages the extracted features for assessment but employs maximum mean discrepancy with a polynomial kernel to measure the distance, showing greater stability in the zero-shot and few-shot conditions. KID, specifically, demonstrates greater stability even with limited samples compared to FID. - NIQE (Natural Image Quality Evaluator)
A no-reference metric, evaluates the distance between the natural scene statistics of distorted images and natural images modeled with a multivariate Gaussian model. - DISTS (Deep Image Structure and Texture Similarity)
DISTS notes that texture and structure similarities between two images can be assessed by their feature means and correlations obtained from VGG and thus utilizes an SSIM-like distance measurement within the feature space to determine texture and structure similarities. - PI
PI is introduced in the PIRM Challenge on perceptual SR, aiming to evaluate the perceptual quality of super-resolved images. Its definition, PI=0.5((10-Ma)+NIQE), incorporates Ma, a no-reference IQA metric for SR.
Reference
Awesome-diffusion-low-level-vision-by-yulunzhang
Awesome-low-level-vision-resources
Awesome-Diffusion-Models-in-Medical-Imaging: Diffusion Models in Medical Imaging