README.md

Datasets

Download our hdf5 datasets (e.g. outdoor_64). The script will first download outdoor_64.zip, and then unzip it into outdoor_64.hdf5).

bash ./datasets/scripts/download_hdf5_dataset.sh outdoor_64

• Outdoor natural images: ourdoor_64.hdf5 (1.4G), outdoor_128.hdf5 (5.5G), 150K landscape images from MIT Places dataset (samples).
• Outdoor church images: church_64.hdf5 (1.3G), church_128.hdf5 (4.6G), 126k church images from the LSUN challenge (samples).
• Shoes images: shoes_64.hdf5 (260MB), shoes_128.hdf5 (922MB), 50K shoes images collected by Yu and Grauman (samples).
• Handbag images: handbag_64.hdf5 (774MB), handbag_128.hdf5 (2.8G), 137K handbag images downloaded from Amazon (samples).

Sketch datasets

Download the sketch datasets (e.g. sketch_shoes_64)

• Shoes sketches: sketch_shoes_64.hdf5 (76MB), sketch_shoes_128.hdf5 (278MB), sketch_shoes_64.hdf5 (69MB), sketch_shoes_128.hdf5 (244MB), 50K shoes sketches collected by Yu and Grauman and filtered by Photoshop sketch filter or HED edge detection. (Photoshop sketch samples, HED samples).

Train a DCGAN model

• Install the following python libraries:

  • Scipy
  • tqdm

  sudo pip install tqdm
  

  • Install h5py and Fuel

  sudo pip install h5py
  sudo pip install git+git://github.com/mila-udem/fuel.git
  

• Train the model with a hdf5 file. (e.g. shoes_64.hdf5)

  • Go the training code directory:

  cd train_dcgan
  

  • Define the model parameters in train_dcgan_config.py file.
  • Train a DCGAN model:

  THEANO_FLAGS='device=gpu0, floatX=float32, nvcc.fastmath=True' python train_dcgan.py --model_name shoes_64
  

  By default, the training code will create a directory ./cache/, and store all the generated samples, webpage, and model checkpoints in the directory.

  • Estimate the batchnorm parameters for DCGAN:

  THEANO_FLAGS='device=gpu0, floatX=float32, nvcc.fastmath=True' python batchnorm_dcgan.py --model_name shoes_64
  

  • (Optionally) Train an inverting model to predict z from x:

  THEANO_FLAGS='device=gpu0, floatX=float32, nvcc.fastmath=True' python train_predict_z.py --model_name shoes_64
  

  • (Optionally) Estaimate the batchnorm parameters for the inverting model:

  THEANO_FLAGS='device=gpu0, floatX=float32, nvcc.fastmath=True' python batchnorm_predict_z.py --model_name shoes_64
  

  • Pack the model:

  python pack_model.py --model_name shoes_64 --output_model shoes_64.dcgan_theano
  

• Train a model on your own dataset

  • Run the script to create a hdf5 file from an image collection (see python create_hdf5.py --help for more details):

  python create_hdf5.py --dataset_dir YOUR_OWN_FOLDER --width 64 --mode dir --channel 3 --hdf5_file images.hdf5
  

Train a generative model (e.g. VAE) based on Theano

The current design of our software follows: ui python class (e.g. gui_draw.py) => python wrapper for constrained optimization (constrained_opt.py) => Theano implementation of constrained optimization (constrained_opt_theano.py) => deep generative model implemented in Theano (e.g. dcgan_theano.py). To incorporate your own generative model, you need to create a new python class (e.g. vae_theano.py) under model_def folder with the same interface of dcgan_theano.py, and specify --model_type vae_theano in the command line.

Train a generative model based on Tensorflow

we are working on a tensorflow based optimization class (i.e. constrained_opt_tensorflow.py) now. Once the code is released, you can create your own tensorflow model class (e.g. dcgan_tensorflow.py) under model_def folder.