Text2Video-Zero

April 5, 2023 ยท View on GitHub

Architecture diagram of Text2Video-Zero

Text2Video-Zero

Implementation of Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators

pip install diffusers==0.14.0 transformers==4.26.0
# ControlNet
pip install git+https://github.com/patrickvonplaten/controlnet_aux.git
python generate.py

Stable Diffusion

Chinese PandaChinese Panda

Version 1
Chinese Panda

Version 2 - Motion in Latents, No Cross-Frame Attention
Chinese Panda a high quality realistic photo of a panda playing guitar on times square

Version 3 - Motion in Latents, Cross-Frame Attention
Chinese Panda a high quality realistic photo of a panda playing guitar on times square an astronaut is skiing down a hill

ControlNet

pose visualization

Contents

  1. 1Stable Diffusion
  2. 2ControlNet