Pyramid Noise

Multi-Resolution Noise for Diffusion Model Training

Abstract

This report proposes a new noising approach that adds multi-resolution noise to an image or latent image during diffusion model training. A model trained with this technique can generate stunning images with a very different aesthetic to the usual diffusion model outputs. This seems like a promising direction for future research.

Citation

Run Training

Run Training

# single gpu
$ diffengine train ${CONFIG_FILE}
# multi gpus
$ NPROC_PER_NODE=${GPU_NUM} diffengine train ${CONFIG_FILE}

# Example.
$ diffengine train stable_diffusion_xl_pokemon_blip_pyramid_noise

Inference with diffusers

You can see details on docs/source/run_guides/run_xl.md.

Results Example

stable_diffusion_xl_pokemon_blip_pyramid_noise