[MM'21] Constrained Graphic Layout Generation via Latent Optimization

October 14, 2022 ยท View on GitHub

This repository provides the official code for the paper "Constrained Graphic Layout Generation via Latent Optimization", especially the code for:

  • LayoutGAN++: generative adversarial networks for layout generation
  • CLG-LO: a framework for generating layouts that satisfy constraints
  • Layout evaluation: measuring the quantitative metrics of Layout FID, Maximum IoU, Alignment, and Overlap for generated layouts

Installation

  1. Clone this repository

    git clone https://github.com/ktrk115/const_layout.git
cd const_layout

  2. Create a new conda environment (Python 3.8)

    conda create -n const_layout python=3.8
conda activate const_layout

  3. Install PyTorch 1.8.1 and PyTorch Geometric 1.7.2. An example of the PyG installation command is shown below.

    pip install torch-scatter==2.0.7 -f https://data.pyg.org/whl/torch-1.8.1+cu111.html
pip install torch-sparse==0.6.10 -f https://data.pyg.org/whl/torch-1.8.1+cu111.html
pip install torch-geometric==1.7.2

  4. Install the other dependent libraries

    pip install -r requirements.txt

  5. Prepare data (see this instruction)

  6. Download pre-trained models

    ./download_model.sh

  7. Install ImageMagick for visualization

Development environment

  • Ubuntu 18.04, CUDA 11.1

LayoutGAN++

Architecture

Training animation

Generate layouts with LayoutGAN++

python generate.py pretrained/layoutganpp_rico.pth.tar --out_path output/generated_layouts.pkl --num_save 5

Train LayoutGAN++ model

python train.py --dataset rico --batch_size 64 --iteration 200000 --latent_size 4 --lr 1e-05 --G_d_model 256 --G_nhead 4 --G_num_layers 8 --D_d_model 256 --D_nhead 4 --D_num_layers 8

CLG-LO

w/ beautification constraintsw/ relational constraints

Generate layouts with beautification constraints

python generate_const.py pretrained/layoutganpp_publaynet.pth.tar --const_type beautify --out_path output/beautify/generated_layouts.pkl --num_save 5

Generate layouts with relational constraints

python generate_const.py pretrained/layoutganpp_publaynet.pth.tar --const_type relation --out_path output/relation/generated_layouts.pkl --num_save 5

Layout evaluation

Evaluate generated layouts

python eval.py rico output/generated_layouts.pkl

A pickle file should be a list of layouts, where each layout is a tuple of bounding boxes and labels. The bounding box is represented by [x, y, width, height] in normalized coordinates, and the label is represented by an index. An example is shown below.

In [x]: layouts
Out[x]:
[(array([[0.47403812, 0.11276676, 0.6250037 , 0.02210438],
         [0.49971417, 0.8550553 , 0.81388366, 0.03492427],
         [0.49919674, 0.47857162, 0.81024694, 0.7070079 ]], dtype=float32),
  array([0, 0, 3]),
  ...

Citation

If this repository helps your research, please consider citing our paper.

@inproceedings{Kikuchi2021,
    title = {Constrained Graphic Layout Generation via Latent Optimization},
    author = {Kotaro Kikuchi and Edgar Simo-Serra and Mayu Otani and Kota Yamaguchi},
    booktitle = {ACM International Conference on Multimedia},
    series = {MM '21},
    year = {2021},
    pages = {88--96},
    doi = {10.1145/3474085.3475497}
}

Licence

GNU AGPLv3