The remarkable progress of generative models has equipped AI systems with human-level capabilities in content generation. Yet, their practical deployment is hindered by catastrophic forgetting—a fundamental issue where learning new tasks erases previously acquired knowledge. Despite growing interest, no comprehensive survey exists to systematically categorize and analyze continual learning methods for mainstream generative models (e.g., Large Language Models, Multimodal Large Language Models, Vision-Language Action Models and Diffusion Models). This work fills the gap by:
- Classifying solutions into architecture-based, regularization-based, and replay-based paradigms, aligning with human-like memory mechanisms.
- Analyzing task adaptations, benchmarks, and model backbones to reveal key insights.
- Prospecting future directions for continual learning in generative models, paving the way for scalable and adaptable intelligence.
- 2026.06: 🔥🔥🔥 Community Highlight: Check out MCITlib, an open-source framework for Multimodal Continual Instruction Tuning. It provides out-of-the-box training and evaluation pipelines for 10+ methods across both image and video modalities, fully compatible with 4 diverse base models.
- 2026.01: We have updated the repository to include relevant papers accepted to ICLR 2026. If you notice any omissions or have any questions, please feel free to open an issue!
- 2025.12: We have released MCITlib, the first complete open-source codebase providing benchmarks and methods for Multimodal Continual Instruction Tuning. The code is open sourced here.
- 2025.07: Check out our new work: "Federated Continual Instruction Tuning" (ICCV 2025). The code is open sourced here.
- 2025.07: We have updated recent public work on continual learning in generative models. If you notice any omissions, please feel free to contact us!
- 2025.06: We released our survey paper "A Comprehensive Survey on Continual Learning in Generative Models". Feel free to cite or open pull requests!
- 2025.06: We released a repository on continual learning in generative models, and a corresponding survey will be available soon.
@article{guo2025comprehensive,
title={A Comprehensive Survey on Continual Learning in Generative Models},
author={Guo, Haiyang and Zeng, Fanhu and Zhu, Fei and Wang, Jiayi and Wang, Xukai and Zhou, Jingang and Zhao, Hongbo and Liu, Wenzhuo and Ma, Shijie and Zhang, Xu-Yao and others},
journal={arXiv preprint arXiv:2506.13045},
year={2025}
}