Awesome Dataset Distillation

June 3, 2026 · View on GitHub

Awesome Contrib PaperNum Stars Forks

Awesome Dataset Distillation provides the most comprehensive and detailed information on the Dataset Distillation field.

Dataset distillation is the task of synthesizing a small dataset such that models trained on it achieve high performance on the original large dataset. A dataset distillation algorithm takes as input a large real dataset to be distilled (training set), and outputs a small synthetic distilled dataset, which is evaluated via testing models trained on this distilled dataset on a separate real dataset (validation/test set). A good small distilled dataset is not only useful in dataset understanding, but has various applications (e.g., continual learning, privacy, neural architecture search, etc.). This task was first introduced in the paper Dataset Distillation [Tongzhou Wang et al., '18], along with a proposed algorithm using backpropagation through optimization steps. Then the task was first extended to the real-world datasets in the paper Medical Dataset Distillation [Guang Li et al., '19], which also explored the privacy preservation possibilities of dataset distillation. In the paper Dataset Condensation [Bo Zhao et al., '20], gradient matching was first introduced and greatly promoted the development of the dataset distillation field.

In recent years (2022-now), dataset distillation has gained increasing attention in the research community, across many institutes and labs. More papers are now being published each year. These wonderful researches have been constantly improving dataset distillation and exploring its various variants and applications.

This project is curated and maintained by Guang Li, Bo Zhao, and Tongzhou Wang.

How to submit a pull request?

  • :globe_with_meridians: Project Page
  • :octocat: Code
  • :book: bibtex

Latest Updates

Contents

Main

Early Work

Gradient/Trajectory Matching Surrogate Objective

Distribution/Feature Matching Surrogate Objective

Kernel-Based Distillation

Distilled Dataset Parametrization

Generative Distillation

GAN

Diffusion

VAR

Flow

Better Optimization

Better Understanding

Label Distillation

Dataset Quantization

Decoupled Distillation

Multimodal Distillation

Self-Supervised Distillation

Benchmark

Survey

Ph.D. Thesis

Workshop

Challenge

Applications

Continual Learning

Privacy

Medical

Federated Learning

Graph Neural Network

Survey

Benchmark

No further updates will be made regarding graph distillation topics as sufficient papers and summary projects are already available on the subject

Fashion, Art, and Design

Recommender Systems

Blackbox Optimization

Robustness

Fairness

Text

Video

Tabular

Retrieval

Domain Adaptation

Super Resolution

Time Series

Speech

Machine Unlearning

Reinforcement Learning

Long-Tail

Learning with Noisy Labels

Object Detection

Point Cloud

Universal Distillation

Spiking Neural Network

EEG

Finance

Music

Remote Sensing

Deraining

Fine-grained

Semantic Segmentation

Media Coverage

Star History

Star History Chart

Citing Awesome Dataset Distillation

If you find this project useful for your research, please use the following BibTeX entry.

@misc{li2022awesome,
  author={Li, Guang and Zhao, Bo and Wang, Tongzhou},
  title={Awesome Dataset Distillation},
  howpublished={\url{https://github.com/Guang000/Awesome-Dataset-Distillation}},
  year={2022}
}

Acknowledgments

We would like to express our heartfelt thanks to Nikolaos Tsilivis, Wei Jin, Yongchao Zhou, Noveen Sachdeva, Can Chen, Guangxiang Zhao, Shiye Lei, Xinchao Wang, Dmitry Medvedev, Seungjae Shin, Jiawei Du, Yidi Jiang, Xindi Wu, Guangyi Liu, Yilun Liu, Kai Wang, Yue Xu, Anjia Cao, Jianyang Gu, Yuanzhen Feng, Peng Sun, Ahmad Sajedi, Zhihao Sui, Ziyu Wang, Haoyang Liu, Eduardo Montesuma, Shengbo Gong, Zheng Zhou, Zhenghao Zhao, Duo Su, Tianhang Zheng, Shijie Ma, Wei Wei, Yantai Yang, Shaobo Wang, Xinhao Zhong, Zhiqiang Shen, Cong Cong, Chun-Yin Huang, Dai Liu, Ruonan Yu, William Holland, Saksham Singh Kushwaha, Ping Liu, Wenliang Zhong, Ning Li, Guochen Yan, Saumyaranjan Mohanty, Taehyung Kwon, Dongwook Kim and Bárbara Capelo for their valuable suggestions and contributions.

The Homepage of Awesome Dataset Distillation was designed by Longzhen Li and maintained by Mingzhuo Li.