Awesome Gen AI for Animation [](https://awesome.re)

January 13, 2026 · View on GitHub

🔥🔥🔥 [ICCV 2025 AISTORY Workshop] Generative AI for Cel-Animation: A Survey

Yunlong Tang1, Junjia Guo1, Pinxin Liu1, Zhiyuan Wang2, Hang Hua1, Jia-Xing Zhong3, Yunzhong Xiao4, Chao Huang1, Luchuan Song1, Susan Liang1, Yizhi Song5, Liu He5, Jing Bi1, Mingqian Feng1, Xinyang Li1, Zeliang Zhang1, Chenliang Xu1

1University of Rochester, 2UCSB, 3University of Oxford, 4CMU, 5Purdue University

ICCV | arXiv | Project Page

📖 Table of Contents

This is the production process of traditional 2D animation. We will list these research topics roughly in this sequence.

Pipeline

A production example showing the transformation of a scene from storyboard to final compositing, demonstrating key stages including layout (L/O), keyframe animation, coloring, and background integration.

News

  • [07/17/2025] Our survey has been accepted by the AISTORY Workshop @ ICCV 2025.
  • [01/08/2025] We release our survey.

Citation

If you find our survey useful, please cite the following paper:

@article{tang2025ai4anime,
     title={Generative AI for Cel-Animation: A Survey},
     author={Tang, Yunlong and Guo, Junjia and Liu, Pinxin and Wang, Zhiyuan and Hua, Hang and Zhong, Jia-Xing and Xiao, Yunzhong and Huang, Chao and Song, Luchuan and Liang, Susan and Song, Yizhi and He, Liu and Bi, Jing and Feng, Mingqian and Li, Xinyang and Zhang, Zeliang and Xu, Chenliang},
     journal={arXiv preprint arXiv:2501.06250},
     url={https://arxiv.org/abs/2501.06250},
     year={2025}
}

🛠️ Methods

1️⃣ Pre-production

📜 Scripting

Model/PaperAuthors/TeamLinksVenue
HoLLMwood: Unleashing the Creativity of Large Language Models in Screenwriting via Role PlayingJing Chen, Xinyu Zhu, Cheng Yang, Chufan Shi, Yadong Xi, Yuxiang Zhang, Junjie Wang, Jiashu Pu, Rongsheng Zhang, Yujiu Yang, Tian Feng
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextGemini Team Google
Claude 3.5 SonnetAnthropic
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic TasksZhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, Jifeng DaiCodeCVPR 2024
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and BeyondJinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, Jingren ZhouCode
GPT-4OpenAI

🎭 Setting

Model/PaperAuthors/TeamLinksVenue
High-Resolution Image Synthesis with Latent Diffusion ModelsRobin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, Björn OmmerDemo CodeCVPR 2022
MidJourneyMidJourney Team

🖌️ Storyboarding

Model/PaperAuthors/TeamLinksVenue
StoryWeaver: A Unified World Model for Knowledge-Enhanced Story Character CustomizationJinlu Zhang, Jiji Tang, Rongsheng Zhang, Tangjie Lv, Xiaoshuai SunCode
Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion ModelsChang Liu, Haoning Wu, Yujie Zhong, Xiaoyun Zhang, Yanfeng Wang, Weidi XieProject Page Code DatasetCVPR 2024
Synthesizing Coherent Story with Auto-Regressive Latent Diffusion ModelsXichen Pan, Pengda Qin, Yuhong Li, Hui Xue, Wenhu ChenCodeWACV 2024
SEED-Story: Multimodal Long Story Generation with Large Language ModelShuai Yang, Yuying Ge, Yang Li, Yukang Chen, Yixiao Ge, Ying Shan, Yingcong ChenCode Dataset
Make-A-Story: Visual Memory Conditioned Consistent Story GenerationTanzila Rahman, Hsin-Ying Lee, Jian Ren, Sergey Tulyakov, Shweta Mahajan, Leonid SigalCode
Make-A-Storyboard: A General Framework for Storyboard with Disentangled and Merged ControlSitong Su, Litao Guo, Lianli Gao, Heng Tao Shen, Jingkuan Song
Animate-A-Story: Storytelling with Retrieval-Augmented Video GenerationYingqing He, Menghan Xia, Haoxin Chen, Xiaodong Cun, Yuan Gong, Jinbo Xing, Yong Zhang, Xintao Wang, Chao Weng, Ying Shan, Qifeng ChenProject Page Code
StoryImager: A Unified and Efficient Framework for Coherent Story Visualization and CompletionMing Tao, Bing-Kun Bao, Hao Tang, Yaowei Wang, Changsheng XuCodeECCV 2024
VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided PlanningHan Lin, Abhay Zala, Jaemin Cho, Mohit BansalProject Page CodeCOLM 2024
CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition AbilitiesTao Wu, Yong Zhang, Xintao Wang, Xianpan Zhou, Guangcong Zheng, Zhongang Qi, Ying Shan, Xi LiProject Page CodeAAAI 2025
VideoStudio: Generating Consistent-Content and Multi-Scene VideosFuchen Long, Zhaofan Qiu, Ting Yao, Tao MeiProject Page CodeECCV 2024
Mind the Time: Temporally-Controlled Multi-Event Video GenerationZiyi Wu, Aliaksandr Siarohin, Willi Menapace, Ivan Skorokhodov, Yuwei Fang, Varnith Chordia, Igor Gilitschenski, Sergey TulyakovProject Page
DreamRunner: Fine-Grained Storytelling Video Generation with Retrieval-Augmented Motion AdaptationZun Wang, Jialu Li, Han Lin, Jaehong Yoon, Mohit BansalProject Page Code
Vlogger: Make Your Dream A VlogShaobin Zhuang, Kunchang Li, Xinyuan Chen, Yaohui Wang, Ziwei Liu, Yu Qiao, Yali WangProject Page CodeCVPR 2024
Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video GenerationYunxin Li, Haoyuan Shi, Baotian Hu, Longyue Wang, Jiashun Zhu, Jinyi Xu, Zhen Zhao, Min ZhangCodeSIGGRAPH Asia 2024
Storyboarder.ai
CogCartoon: Towards Practical Story VisualizationZhongyang Zhu, Jie Tang

2️⃣ Production

🗺️ Layout (L/O)

Model/PaperAuthors/TeamLinksVenue
CogCartoon: Towards Practical Story VisualizationZhongyang Zhu, Jie Tang
Sketch-Guided Scene Image GenerationTianyu Zhang, Xiaoxuan Xie, Xusheng Du, Haoran Xie
VideoComposer: Compositional Video Synthesis with Motion ControllabilityXiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Jiuniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, Jingren ZhouProject Page CodeNIPS 2023
LayoutGAN: Generating Graphic Layouts with Wireframe DiscriminatorsJianan Li, Jimei Yang, Aaron Hertzmann, Jianming Zhang, Tingfa XuCodeICLR 2019
VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided PlanningHan Lin, Abhay Zala, Jaemin Cho, Mohit BansalProject Page CodeCOLM 2024
DiffSensei: Bridging Multi-Modal LLMs and Diffusion Models for Customized Manga GenerationJianzong Wu, Chao Tang, Jingbo Wang, Yanhong Zeng, Xiangtai Li, Yunhai TongProject Page Code Dataset
Manga Generation via Layout-controllable DiffusionSiyu Chen, Dengjie Li, Zenghao Bao, Yao Zhou, Lingfeng Tan, Yujie Zhong, Zheng ZhaoProject Page Code
CameraCtrl: Enabling Camera Control for Text-to-Video GenerationHao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, Ceyuan YangProject Page Demo

🎞️ Keyframe Animation

Model/PaperAuthors/TeamLinksVenue
Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character AnimationCVPR 2024Project Page CodeCVPR 2024
Champ: Controllable and Consistent Human Image Animation with 3D Parametric GuidanceShenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Qingkun Su, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, Siyu ZhuProject Page Code
MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose GuidanceYuang Zhang, Jiaxi Gu, Li-Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, Fangyuan ZouProject Page Code
Animate-X: Universal Character Image Animation with Enhanced Motion RepresentationShuai Tan, Biao Gong, Xiang Wang, Shiwei Zhang, Dandan Zheng, Ruobing Zheng, Kecheng Zheng, Jingdong Chen, Ming YangCode
MikuDance: Animating Character Art with Mixed Motion DynamicsJiaxu Zhang, Xianfang Zeng, Xin Chen, Wei Zuo, Gang Yu, Zhigang TuProject Page Code
Collaborative Neural Rendering using Anime Character SheetsZuzeng Lin, Ailin Huang, Zhewei HuangCode DatasetIJCAI 2023
Textoon: Generating Vivid 2D Cartoon Characters from Text DescriptionsChao He, Jianqiang Ren, Liefeng BoProject Page Code
AnimeGamer: Infinite Anime Life Simulation with Next Game State PredictionJunhao Cheng, Yuying Ge, Yixiao Ge, Jing Liao, Ying ShanProject Page Code

From: ToonComposer: Streamlining Cartoon Production with Generative Post-Keyframing

🍴 Inbetweening

Model/PaperAuthors/TeamLinksVenue
ToonComposer: Streamlining Cartoon Production with Generative Post-KeyframingLingen Li, Guangzhi Wang, Zhaoyang Zhang, Yaowei Li, Xiaoyu Li, Qi Dou, Jinwei Gu, Tianfan Xue, Ying ShanProject
ToonCrafter: Generative Cartoon InterpolationJinbo Xing, Hanyuan Liu, Menghan Xia, Yong Zhang, Xintao Wang, Ying Shan, Tien-Tsin WongCode ProjectarXiv 2024
Joint Stroke Tracing and Correspondence for 2D AnimationHaoran Mo, Chengying Gao, Ruomei WangProject CodeSIGGRAPH 2024
Deep Geometrized Cartoon Line InbetweeningLi Siyao, Tianpei Gu, Weiye Xiao, Henghui Ding, Ziwei Liu, Chen Change LoyDemo Code DatasetICCV 2023
Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout AnimationT Fukusato, A Maejima, T Igarashi, T YotsukuraMTA 2023
Enhanced Deep Animation Video InterpolationWang Shen, Cheng Ming, Wenbo Bao, Guangtao Zhai, Li Chen, Zhiyong GaoarXiv 2022
Improving the Perceptual Quality of 2D Animation InterpolationShuhong Chen, Matthias ZwickerarXiv 2021
Deep Animation Video Interpolation in the WildLi Siyao, Shiyu Zhao, Weijiang Yu, Wenxiu Sun, Dimitris N. Metaxas, Chen Change Loy, Ziwei LiuCode DatasetarXiv 2021
Deep Sketch-Guided Cartoon Video InbetweeningXiaoyu Li, Bo Zhang, Jing Liao, Pedro V. SanderarXiv 2020
Optical Flow Based Line Drawing Frame Interpolation Using Distance Transform to Support InbetweeningsRei Narita, Keigo Hirakawa, Kiyoharu AizawaIEEE 2019
DiLight: Digital Light Table – Inbetweening for 2D Animations Using GuidelinesLeonardo Carvalho, Ricardo Marroquim, Emilio Vital BrazilElsevier 2017
Anisora: Exploring the frontiers of animation video generation in the sora eraYudong Jiang, Baohan Xu, Siqian Yang, Mingyu Yin, Jing Liu, Chao Xu, Siqi Wang, Yidi Wu, Bingwen Zhu, Xinwen Zhang, Xingyu Zheng, Jixuan Xu, Yue Zhang, Jinlong Hou, Huyang SunCode
LayerAnimate: Layer-specific Control for AnimationYuxue Yang, Lue Fan, Zuzeng Lin, Feng Wang, Zhaoxiang ZhangProject Page Code
Framer: Interactive Frame InterpolationWen Wang, Qiuyu Wang, Kecheng Zheng, Hao Ouyang, Zhekai Chen, Biao Gong, Hao Chen, Yujun Shen, Chunhua ShenProject Page Code Demo

From: ToonCrafter: Generative Cartoon Interpolation

🌈 Colorization

Model/PaperAuthors/TeamLinksVenue
ToonComposer: Streamlining Cartoon Production with Generative Post-KeyframingLingen Li, Guangzhi Wang, Zhaoyang Zhang, Yaowei Li, Xiaoyu Li, Qi Dou, Jinwei Gu, Tianfan Xue, Ying ShanProject
Learning Inclusion Matching for Animation Paint Bucket ColorizationYuekun Dai, Shangchen Zhou, Qinyue Li, Chongyi Li, Chen Change LoyProject Demo Code DatasetCVPR 2024
AniDoc: Animation Creation Made EasierYihao Meng, Hao Ouyang, Hanlin Wang, Qiuyu Wang, Wen Wang, Ka Leong Cheng, Zhiheng Liu, Yujun Shen, Huamin QuProject CodearXiv 2024
ToonCrafter: Generative Cartoon InterpolationJinbo Xing, Hanyuan Liu, Menghan Xia, Yong Zhang, Xintao Wang, Ying Shan, Tien-Tsin WongProject CodeTOG 2024
VToonify: Controllable High-Resolution Portrait Video Style TransferShuai Yang, Liming Jiang, Ziwei Liu, Chen Change LoyProject CodeTOG 2022
StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned FacesShuai Yang, Liming Jiang, Ziwei Liu, Chen Change LoyProject Code DemoICCV 2023
FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video TranslationShuai Yang, Yifan Zhou, Ziwei Liu, Chen Change LoyProject Code DemoCVPR 2024
TokenFlow: Consistent Diffusion Features for Consistent Video EditingMichal Geyer, Omer Bar-Tal, Shai Bagon, Tali DekelyProject Code DemoICLR 2024
PromptFix: You Prompt and We Fix the PhotoYongsheng Yu, Ziyun Zeng, Hang Hua, Jianlong Fu, Jiebo LuoProject CodeNIPS 2024
LVCD: Reference-based Lineart Video Colorization with Diffusion ModelsZhitong Huang, Mohan Zhang, Jing LiaoProject CodeTOG 2024
Coloring Anime Line Art Videos with Transformation Region Enhancement NetworkNing Wang, Muyao Niu, Zhi Dou, Zhihui Wang, Zhiyong Wang, Zhaoyan Ming, Bin Liu, Haojie LiPattern Recognition 2023
SketchBetween: Video-to-Video Synthesis for Sprite Animation via SketchesDagmar Lukka Loftsdóttir, Matthew GuzdialCodeECCV 2022
Animation Line Art Colorization Based on Optical Flow MethodYifeng Yu, Jiangbo Qian, Chong Wang, Yihong Dong, Baisong LiuSSNR 2022
The Animation Transformer: Visual Correspondence via Segment MatchingEvan Casey, Víctor Pérez, Zhuoru Li, Harry Teitelman, Nick Boyajian, Tim Pulver, Mike Manh, William GrisaitisDemoarXiv 2021
Artist-Guided Semiautomatic Animation ColorizationHarrish Thasarathan, Mehran EbrahimiarXiv 2020
Line Art Correlation Matching Feature Transfer Network for Automatic Animation ColorizationZhang Qian, Wang Bo, Wen Wei, Li Hai, Liu Jun HuiarXiv 2020
Deep Line Art Video Colorization with a Few ReferencesMin Shi, Jia-Qi Zhang, Shu-Yu Chen, Lin Gao, Yu-Kun Lai, Fang-Lue ZhangarXiv 2020
Automatic Temporally Coherent Video ColorizationHarrish Thasarathan, Kamyar Nazeri, Mehran EbrahimiCodearXiv 2019
ToonaToona Team
LayerAnimate: Layer-specific Control for AnimationYuxue Yang, Lue Fan, Zuzeng Lin, Feng Wang, Zhaoxiang ZhangProject Page Code
MangaNinja: Line Art Colorization with Precise Reference FollowingZhiheng Liu, Ka Leong Cheng, Xi Chen, Jie Xiao, Hao Ouyang, Kai Zhu, Yu Liu, Yujun Shen, Qifeng Chen, Ping LuoProject Page Code

From: Learning Inclusion Matching for Animation Paint Bucket Colorization

3️⃣ Post-Production

📷 Compositing & Photography

Model/PaperAuthors/TeamLinksVenue
ToonComposer: Streamlining Cartoon Production with Generative Post-KeyframingLingen Li, Guangzhi Wang, Zhaoyang Zhang, Yaowei Li, Xiaoyu Li, Qi Dou, Jinwei Gu, Tianfan Xue, Ying ShanProject
Scaling In-the-Wild Training for Diffusion-based Illumination Harmonization and Editing by Imposing Consistent Light Transport
DoveNet: Deep Image Harmonization via Domain VerificationWenyan Cong, Jianfu Zhang, Li Niu, Liu Liu, Zhixin Ling, Weiyuan Li, Liqing ZhangCode Demo Dataset(Baidu Cloud(access code: kqz3)) Dataset(OneDrive)CVPR 2020
High-Resolution Image Harmonization via Collaborative Dual TransformationsWenyan Cong, Xinhao Tao, Li Niu, Jing Liang, Xuesong Gao, Qihao Sun, Liqing ZhangCodeCVPR 2022
PCT-Net: Full Resolution Image Harmonization Using Pixel-Wise Color TransformationsJulian Jorge Andrade Guerreiro, Mitsuru Nakazawa, Björn StengerCodeCVPR 2023
SSH: A Self-Supervised Framework for Image HarmonizationYifan Jiang, He Zhang, Jianming Zhang, Yilin Wang, Zhe Lin, Kalyan Sunkavalli, Simon Chen, Sohrab Amirghodsi, Sarah Kong, Zhangyang WangCodeICCV 2021
Thinking Outside the BBox: Unconstrained Generative Object CompositingGemma Canet Tarrés, Zhe Lin, Zhifei Zhang, Jianming Zhang, Yizhi Song, Dan Ruta, Andrew Gilbert, John Collomosse, Soo Ye KimECCV 2024
Dr.Bokeh: DiffeRentiable Occlusion-aware Bokeh RenderingYichen Sheng, Zixun Yu, Lu Ling, Zhiwen Cao, Cecilia Zhang, Xin Lu, Ke Xian, Haiting Lin, Bedrich BenesCode
Floating No More: Object-Ground Reconstruction from a Single ImageYunze Man, Yichen Sheng, Jianming Zhang, Liang-Yan Gui, Yu-Xiong WangProject Page
ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and InsertionDaniel Winter, Matan Cohen, Shlomi Fruchter, Yael Pritch, Alex Rav-Acha, Yedid HoshenProject Page
Alchemist: Parametric Control of Material Properties with Diffusion ModelsPrafull Sharma, Varun Jampani, Yuanzhen Li, Xuhui Jia, Dmitry Lagun, Fredo Durand, William T. Freeman, Mark MatthewsProject PageCVPR 2024
DisenStudio: Customized Multi-Subject Text-to-Video Generation with Disentangled Spatial ControlHong Chen, Xin Wang, Yipeng Zhang, Yuwei Zhou, Zeyang Zhang, Siao Tang, Wenwu ZhuCodeACMMM 2024
SSN: Soft Shadow Network for Image CompositingYichen Sheng, Jianming Zhang, Bedrich BenesProject Page CodeCVPR 2021
LayerAnimate: Layer-specific Control for AnimationYuxue Yang, Lue Fan, Zuzeng Lin, Feng Wang, Zhaoxiang ZhangProject Page Code

✂️ Cutting (CT)

Model/PaperAuthors/TeamLinksVenue
Multi-modal Segment Assemblage Network for Ad Video Editing with Importance-Coherence RewardYunlong Tang, Siting Xu, Teng Wang, Qin Lin, Qinglin Lu, Feng ZhengCodeACCV 2022
Reframe Anything: LLM Agent for Open World Video ReframingJiawang Cao, Yongliang Wu, Weiheng Chi, Wenbo Zhu, Ziyue Su, Jay Wu
OpusClipOpusClip Team

🎶 Music & Sound Effects

Model/PaperAuthors/TeamLinksVenue
Foley Music: Learning to Generate Music from VideosChuang Gan, Deng Huang, Peihao Chen, Joshua B. Tenenbaum, Antonio TorralbaCodeECCV 2020
Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer modelDemo Project Page Code Dataset
V2Meow: Meowing to the Visual Beat via Video-to-Music GenerationKun Su, Judith Yue Li, Qingqing Huang, Dima Kuzmin, Joonseok Lee, Chris Donahue, Fei Sha, Aren Jansen, Yu Wang, Mauro Verzetti, Timo I. DenkProject PageAAAI 2024
MeLFusion: Synthesizing Music from Image and Language Cues using Diffusion ModelsSanjoy Chowdhury, Sayan Nag, K J Joseph, Balaji Vasan Srinivasan, Dinesh ManochaProject Page Code DatasetCVPR 2024
VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term ModelingZeyue Tian, Zhaoyang Liu, Ruibin Yuan, Jiahao Pan, Qifeng Liu, Xu Tan, Qifeng Chen, Wei Xue, Yike GuoCode
Taming Visually Guided Sound GenerationVladimir Iashin, Esa RahtuProject Page Demo CodeBMVC 2021
I Hear Your True Colors: Image Guided Audio GenerationRoy Sheffer, Yossi AdiProject Page CodeICASSP 2023
FoleyGen: Visually-Guided Audio GenerationXinhao Mei, Varun Nagaraja, Gael Le Lan, Zhaoheng Ni, Ernie Chang, Yangyang Shi, Vikas ChandraProject Page
Diff-Foley: Synchronized Video-to-Audio Synthesis with Latent Diffusion ModelsSimian Luo, Chuanhao Yan, Chenxu Hu, Hang ZhaoProject Page CodeNeurIPS 2023
Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric VideosChangan Chen, Puyuan Peng, Ami Baid, Zihui Xue, Wei-Ning Hsu, David Harwath, Kristen GraumanProject Page CodeECCV 2024

🎙️ After Record (AR) & Dubbing (DB)

Model/PaperAuthors/TeamLinksVenue
StyleDubber: Towards Multi-Scale Style Learning for Movie DubbingGaoxiang Cong, Yuankai Qi, Liang Li, Amin Beheshti, Zhedong Zhang, Anton van den Hengel, Ming-Hsuan Yang, Chenggang Yan, Qingming HuangCodeACL 2024
ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of VideoKevin Cai, Chonghua Liu, David M. ChanCode DatasetICASSP 2024
EmoDubber: Towards High Quality and Emotion Controllable Movie DubbingGaoxiang Cong, Jiadong Pan, Liang Li, Yuankai Qi, Yuxin Peng, Anton van den Hengel, Jian Yang, Qingming HuangProject Page & Demo
From Speaker to Dubber: Movie Dubbing with Prosody and Duration Consistency LearningZhedong Zhang, Liang Li, Gaoxiang Cong, Haibing Yin, Yuhan Gao, Chenggang Yan, Anton van den Hengel, Yuankai QiCodeACM MM 2024
Learning to Dub Movies via Hierarchical Prosody ModelsGaoxiang Cong, Liang Li, Yuankai Qi, Zhengjun Zha, Qi Wu, Wenyu Wang, Bin Jiang, Ming-Hsuan Yang, Qingming HuangCodeCVPR 2023
V2C: Visual Voice CloningQi Chen, Yuanqing Li, Yuankai Qi, Jiaqiu Zhou, Mingkui Tan, Qi WuCodeCVPR 2022

Others

🎞️ Cel-Animation Editing

Model/PaperAuthors/TeamLinksVenue
Re:Draw -- Context Aware Translation as a Controllable Method for Artistic ProductionJoao Liborio Cardoso, Francesco Banterle, Paolo Cignoni, Michael WimmerTBA 2024
Scaling Concept With Text-Guided Diffusion ModelsChao Huang, Susan Liang, Yunlong Tang, Yapeng Tian, Anurag Kumar, Chenliang XuProject Page CodeICLR 2025

🎨 Cels Decomposition

Model/PaperAuthors/TeamLinksVenue
Sprite-from-Sprite: Cartoon Animation Decomposition with Self-supervised Sprite EstimationLvmin Zhang, Tien-Tsin Wong, Yuxin LiuCodeACM 2022
Generative Omnimatte: Learning to Decompose Video into LayersYao-Chih Lee, Erika Lu, Sarah Rumbley, Michal Geyer, Jia-Bin Huang, Tali Dekel, Forrester ColeProject Page
LayerAnimate: Layer-specific Control for AnimationYuxue Yang, Lue Fan, Zuzeng Lin, Feng Wang, Zhaoxiang ZhangProject Page Code
TransPixar: Advancing Text-to-Video Generation with TransparencyLuozhou Wang, Yijun Li, Zhifei Chen, Jui-Hsien Wang, Zhifei Zhang, He Zhang, Zhe Lin, Yingcong ChenProject Page Code

🏯 3D Assistance

Model/PaperAuthors/TeamLinksVenue
Toonsynth: Example-based Synthesis of Hand-Colored Cartoon AnimationsM Dvorožnák, W Li, VG Kim, D SýkoraTOG 2018
Collaborative Neural Rendering using Anime Character SheetsZuzeng Lin, Ailin Huang, Zhewei HuangCode DatasetIJCAI 2023
DrawingSpinUp: 3D Animation from Single Character DrawingsJie Zhou, Chufeng Xiao, Miu-Ling Lam, Hongbo FuCodeSiggraph Asia 2024

📊 Datasets

Model/PaperAuthors/TeamLinksVenue
Sakuga-42M Dataset: Scaling Up Cartoon ResearchZhenglin Pan, Yu Zhu, Yuxuan MuDatasetarXiv 2024
Anisora: Exploring the frontiers of animation video generation in the sora eraYudong Jiang, Baohan Xu, Siqian Yang, Mingyu Yin, Jing Liu, Chao Xu, Siqi Wang, Yidi Wu, Bingwen Zhu, Xinwen Zhang, Xingyu Zheng, Jixuan Xu, Yue Zhang, Jinlong Hou, Huyang SunCode
ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of VideoKevin Cai, Chonghua Liu, David M. ChanCode DatasetICASSP 2024
V2C: Visual Voice CloningQi Chen, Yuanqing Li, Yuankai Qi, Jiaqiu Zhou, Mingkui Tan, Qi WuCodeCVPR 2022
DoveNet: Deep Image Harmonization via Domain VerificationWenyan Cong, Jianfu Zhang, Li Niu, Liu Liu, Zhixin Ling, Weiyuan Li, Liqing ZhangCode Demo Dataset (Baidu Cloud) Dataset (OneDrive)CVPR 2020
SSH: A Self-Supervised Framework for Image HarmonizationYifan Jiang, He Zhang, Jianming Zhang, Yilin Wang, Zhe Lin, Kalyan Sunkavalli, Simon Chen, Sohrab Amirghodsi, Sarah Kong, Zhangyang WangCode DatasetICCV 2021
Intrinsic Image HarmonizationZonghui Guo, Haiyong Zheng, Yufeng Jiang, Zhaorui Gu, Bing ZhengCode Dataset (Baidu Cloud) Dataset (Google Drive)CVPR 2021
Alchemist: Parametric Control of Material Properties with Diffusion ModelsPrafull Sharma, Varun Jampani, Yuanzhen Li, Xuhui Jia, Dmitry Lagun, Fredo Durand, William T. Freeman, Mark MatthewsProject PageCVPR 2024
Learning Inclusion Matching for Animation Paint Bucket ColorizationYuekun Dai, Shangchen Zhou, Qinyue Li, Chongyi Li, Chen Change LoyProject Demo Code DatasetCVPR 2024
Deep Animation Video Interpolation in the WildLi Siyao, Shiyu Zhao, Weijiang Yu, Wenxiu Sun, Dimitris N. Metaxas, Chen Change Loy, Ziwei LiuCode Data (Google Drive) Data (OneDrive) Video DemoCVPR 2021
Deep Geometrized Cartoon Line InbetweeningLi Siyao, Tianpei Gu, Weiye Xiao, Henghui Ding, Ziwei Liu, Chen Change LoyDemo Code DatasetICCV 2023
AnimeRun: 2D Animation Visual Correspondence from Open Source 3D MoviesLi Siyao, Yuhang Li, Bo Li, Chao Dong, Ziwei Liu, Chen Change LoyProject & Dataset CodeNeurIPS 2022

From: Sakuga-42M Dataset: Scaling Up Cartoon Research

🌟 Star History

Star History Chart

♥️ Contributors

Our project wouldn't be possible without the contributions of these amazing people! Thank you all for making this project better.

Yunlong Tang @ University of Rochester
Junjia Guo @ University of Rochester
Pinxin Liu @ University of Rochester
Zhiyuan Wang @ UCSB
Hang Hua @ University of Rochester
Jia-Xing Zhong @ University of Oxford
Yunzhong Xiao @ CMU
Chao Huang @ University of Rochester
Luchuan Song @ University of Rochester
Susan Liang @ University of Rochester
Yizhi Song @ Purdue University
Liu He @ Purdue University
Jing Bi @ University of Rochester
Mingqian Feng @ University of Rochester
Xinyang Li @ University of Rochester
Zeliang Zhang @ University of Rochester
Chenliang Xu @ University of Rochester