The Lottery Ticket Hypothesis in Denoising: Towards Semantic-Driven Initialization (ECCV 2024)

[Project Page] [Paper]

Refer to our previous work for more discussion about initial noise in diffusion!

• Guided Image Synthesis via Initial Image Editing in Diffusion Model (ACM MM 2023)

Setup

Our codebase is built on CompVis/stable-diffusion and has shared dependencies and model architecture.

Creating a Conda Environment

conda env create -f environment.yaml
conda activate ldm

Downloading StableDiffusion Weights

Download the StableDiffusion weights from the CompVis organization at Hugging Face (download the sd-v1-4.ckpt file), and link them:

mkdir -p models/ldm/stable-diffusion-v1/
ln -s <path/to/model.ckpt> models/ldm/stable-diffusion-v1/model.ckpt 

Hands on

Play with hands-on to try our approach right away, refer to utils.py for the implementation.

Citation

@article{mao2024theLottery,
  title={The Lottery Ticket Hypothesis in Denoising: Towards Semantic-Driven Initialization},
  author={Mao, Jiafeng and Wang, Xueting and Aizawa, Kiyoharu},
  journal={ECCV},
  year={2024}
}