Instructions to train models and baselines

Prerequisites

Make sure the data loading and preprocessing is complete. The training will also be visualized using visdom which can be started by

python -m visdom.server

Note that all the training jobs should be launched from one directory above CODE_ROOT. Additionally, the sample scripts below assume that the code folder is named 'factored3d'.

Training factored 3D prediction models

We first train the object-centric 3D prediction module. Since training with proposals or predicting full voxels is computationally expensive, we train in stages to speed up the process.

# Download a pretrained object voxel auto-encoder
cd CODE_ROOT/cachedir/snapshots;
wget https://people.eecs.berkeley.edu/~shubhtuls/cachedir/factored3d/object_autoenc_32.tar.gz && tar -xvzf object_autoenc_32.tar.gz

# All jobs should be launched from one level above code directory
cd CODE_ROOT/..;

# First train the object-centric 3D prediction model on ground-truth boxes
python -m factored3d.experiments.suncg.box3d --plot_scalars --display_visuals --display_freq=2000 --save_epoch_freq=1 --batch_size=8  --name=box3d_base --use_context --pred_voxels=False --classify_rot --shape_loss_wt=10 --n_data_workers=0 --num_epochs=8

# Fine-tune the above model using proposals
python -m factored3d.experiments.suncg.dwr --name=dwr_base --classify_rot --pred_voxels=False --use_context --plot_scalars --display_visuals --save_epoch_freq=1 --display_freq=1000 --display_id=100 --box3d_ft --shape_loss_wt=10 --label_loss_wt=10  --batch_size=8 --num_epochs=1

# Finally, also learn to predict shape voxels instead of auto-encoder shape code
python -m factored3d.experiments.suncg.dwr --name=dwr_shape_ft --classify_rot --pred_voxels=True --shape_dec_ft --use_context --plot_scalars --display_visuals --save_epoch_freq=1 --display_freq=1000 --display_id=100 --shape_loss_wt=2  --label_loss_wt=10 --batch_size=8 --ft_pretrain_epoch=1 --num_epochs=1

We also train the layout (amodal inverse depth) prediction CNN

# job should be launched from one level above code directory
cd CODE_ROOT/..

python -m factored3d.experiments.suncg.layout --plot_scalars --display_visuals --save_epoch_freq=1 --batch_size=8 --name=layout_pred --display_freq=2000 --suncg_dl_out_layout=true --suncg_dl_out_depth=false --display_id=40 --num_epochs=8

Training (inverse) depth prediction baseline

python -m factored3d.experiments.suncg.layout --plot_scalars --display_visuals --save_epoch_freq=1 --batch_size=8 --name=depth_baseline --display_freq=2000 --suncg_dl_out_layout=false --suncg_dl_out_depth=true --display_id=20 --num_epochs=8

Training scene voxel prediction baseline

# job should be launched from one level above code directory
cd CODE_ROOT/..;

python -m factored3d.experiments.suncg.voxels --plot_scalars --display_visuals --save_epoch_freq=1 --batch_size=8 --name=voxels_baseline --display_freq=2000 --num_epochs=8