Results on Structured3D dataset

November 27, 2019 · View on GitHub

References:

Dataset preparation

Please contact Structured3D to get the datas.
Download all zip files under {YOUR_DIR}. Don't extract them.

Run below to extract rgb and layout with original furniture and lighting setting only:

python misc/structured3d_extract_zip.py --zippath {YOUR_DIR}/Structured3D_0.zip --outdir {TARGET_DIR_EXTRACT}
python misc/structured3d_extract_zip.py --zippath {YOUR_DIR}/Structured3D_1.zip --outdir {TARGET_DIR_EXTRACT}
# ... for all Structured3D_?.zip

Run below to create data/st3d_[train|valid|test]_full_raw_light following HorizonNet training and testing dataset format. (This will use soft link instead of copy a new one.)
```
python misc/structured3d_prepare_dataset.py --in_root {TARGET_DIR_EXTRACT}
```

Training

python train.py --train_root_dir data/st3d_train_full_raw_light/ --valid_root_dir data/st3d_valid_full_raw_light/ --id resnet50_rnn__st3d --lr 3e-4 --batch_size_train 24 --epochs 50

See python train.py -h or README.md#training for more detail.

Download the trained model: resnet50_rnn__st3d.pth.

Trained on Structured3D 18362 pano images with setting of original furniture and lighting.
Trained for 50 epoch.
Select 50th epoch according to loss function on validation set.

Testing

Generating layout for testing set:

python inference.py --pth ckpt/resnet50_rnn__st3d.pth --img_glob "data/st3d_test_full_raw_light/img/*" --output_dir ./output/st3d/resnet50_rnn/ --visualize

--output_dir: a directory you want to dump the extracted layout
--visualize: visualize raw output (without post-processing) from HorizonNet.

Quantitative evaluatation:

python eval_general.py --dt_glob "./output/st3d/resnet50_rnn/*json" --gt_glob "data/st3d_test_full_raw_light/label_cor/*"

:clipboard: Below is the quantitative result on Structured3D testing set.

# of corners	instances	3D IoU	2D IoU
4	1067	`94.14`	`95.50`
6	290	`90.34`	`91.54`
8	130	`87.98`	`89.43`
10+	202	`79.95`	`81.10`
odd	4	`88.62`	`89.80`
overall	1693	`91.31`	`92.63`

The odd row mean non-even number of corners (ground truth is obviously non-manhattan layout while model output is the approximation of it)

Four instances are skip by eval_general.py as the ground truth is self-intersecting. The top-down view of the four skipped instance are illutrated below where the red dot line are estimated by HorizonNet and the green solid line is the self-intersected ground truth.