Getting Started

August 10, 2023 · View on GitHub

This document provides a brief intro of the usage of DAPS3D.

Training

Make shure to cteate conda environment and setup wandb.login for training (See Installation).
Select desired augmentation setup inside augmentation.yml. You can find more detailed description and different augmentation setups in our Paper.

Augmentations

We have different configs for our models: salsanext.yml, ddrnet23_slim.yml & segformer.yml. You can change hyperparameters there before training.

Train Commands

This is default command structure for training:

./train.sh -d <path/to/dataset> \
           -f <path/to/configs> \
           -a <path/to/model_config> \
           -m <model_name> \
           -l <path/to/save/logs> \
           -p <path/to/pretrained/logs> \
           -c <gpu to run>

You need to choose <model_name> between: salsanet, salsanet_rec, salsanet_rec_lstm, salsanext, salsanext_rec_lstm, ddrnet or segformer.

SalsaNet

With default config requires 6 GB on GPU

./train.sh -d /Dataset/ -f cfgs/ -a cfgs/salsanext.yml -m salsanet -l ./logs/ -c 0

SalsaNetRec

With default config requires 7 GB on GPU

./train.sh -d /Dataset/ -f cfgs/ -a cfgs/salsanext.yml -m salsanet_rec -l ./logs/ -c 0

SalsaNetRecLSTM

With default config requires 9 GB on GPU

./train.sh -d /Dataset/ -f cfgs/ -a cfgs/salsanext.yml -m salsanet_rec_lstm -l ./logs/ -c 0

SalsaNext

With default config requires 10 GB on GPU

./train.sh -d /Dataset/ -f cfgs/ -a cfgs/salsanext.yml -m salsanext -l ./logs/ -c 0

SalsaNextRecLSTM

With default config requires 14 GB on GPU

./train.sh -d /Dataset/ -f cfgs/ -a cfgs/salsanext.yml -m salsanext_rec_lstm -l ./logs/ -c 0

DDRNet

Change MODEL.MOD inside ddrnet23_slim.yml to 'none', 'oc' or 'da' for different model configurations before run.

With default config requires 2, 3 & 10 GB on GPU for none, oc & da respectively

./train.sh -d /Dataset/ -f cfgs/ -a cfgs/ddrnet23_slim.yml -m ddrnet -l ./logs/ -c 0

SegFormer

With default config requires 11 GB on GPU

./train.sh -d /Dataset/ -f cfgs/ -a cfgs/segformer.yml -m segformer -l ./logs/ -c 0

Inference

This is default command structure for inference:

./infer.sh -d <path/to/dataset> \
          -f <path/to/configs> \
          -l <path/to/pretrained/logs> \
          -m <model_name> \
          -p <path/to/save/predictions> \
          -s <split to inference> \
          -c <gpu to run>

Simple example (for DDRNet model):

./infer.sh -d /Dataset/ -f cfgs/ -l ./logs/ddrnet_aug-set-5+t-z/ -m ddrnet  -p ./logs/infer/ddrnet_aug-set-5+t-z -s valid -c 0

In order to calculate metrics for your predicrions, run eval.sh. Metrics will be saved in iou.txt inside your <predictions folder>:

./eval.sh -d /Dataset/ -f cfgs/ -p ./logs/infer/ddrnet_aug-set-5+t-z -s valid

Pretrained Models

Segmentation models trained and evaluated on SemanticKITTI dataset

All models are trained with augmentation set 5 without T-Zone (see our Paper)

Model	$mIoU$ , %	$IoU$ [vehicle], %	$IoU$ [human], %	$IoU$ [surface], %	$IoU$ [obstacle], %	Checkpoint
SalsaNet	0.787	0.882	0.412	0.929	0.924	model
SalsaNetRec	0.789	0.855	0.488	0.913	0.900	model
SalsaNetRecLSTM	0.751	0.887	0.271	0.927	0.920	model
SalsaNext	0.821	0.907	0.564	0.905	0.907	model
SalsaNextRecLSTM	0.835	0.914	0.600	0.911	0.914	model
DDRNet	0.692	0.750	0.225	0.901	0.893	model
DDRNetOC	0.687	0.739	0.222	0.900	0.889	model
DDRNetDA	0.696	0.754	0.232	0.903	0.895	model
SegFormer	0.539	0.437	0.048	0.893	0.777	model

Segmentation models trained and evaluated on SemanticKITTI and RELLIS-3D datasets

All models are trained with augmentation set 5 with T-Zone (see our Paper)

Model	$mIoU$ , %	$IoU$ [vehicle], %	$IoU$ [human], %	$IoU$ [surface], %	$IoU$ [obstacle], %	Checkpoint
SalsaNet	0.832	0.869	0.886	0.789	0.782	model
SalsaNetRec	0.808	0.858	0.847	0.763	0.766	model
SalsaNetRecLSTM	0.828	0.889	0.880	0.770	0.774	model
SalsaNext	0.832	0.904	0.906	0.755	0.763	model
SalsaNextRecLSTM	0.833	0.908	0.916	0.748	0.760	model
DDRNet	0.706	0.759	0.654	0.685	0.725	model
DDRNetOC	0.705	0.749	0.652	0.693	0.728	model
DDRNetDA	0.658	0.715	0.471	0.716	0.731	model
SegFormer	0.533	0.448	0.423	0.616	0.643	model

Segmentation models trained and evaluated on DAPS-1 dataset

All models are trained with augmentation set 5 with T-Zone (see our Paper)

Model	$mIoU$ , %	$IoU$ [vehicle], %	$IoU$ [human], %	$IoU$ [surface], %	$IoU$ [obstacle], %	Checkpoint
SalsaNet	0.867	0.880	0.646	0.989	0.953	model
SalsaNetRec	0.850	0.836	0.680	0.974	0.908	model
SalsaNetRecLSTM	0.862	0.878	0.632	0.988	0.949	model
SalsaNext	0.886	0.878	0.721	0.990	0.954	model
SalsaNextRecLSTM	0.932	0.929	0.830	0.994	0.974	model
DDRNet	0.690	0.773	0.126	0.977	0.886	model
DDRNetOC	0.694	0.769	0.138	0.978	0.889	model
DDRNetDA	0.691	0.770	0.129	0.977	0.886	model
Segformer	0.530	0.495	0.052	0.909	0.665	model

Segmentation models evaluated on DAPS-2 dataset

These are inference results on DAPS-2 for the models listed in the following sections (see our Paper)

Models trained on SemanticKITTI

Model	$mIoU$ , %	$IoU$ [human], %	$IoU$ [surface], %	$IoU$ [obstacle], %	Checkpoint
SalsaNet	0.405	0.056	0.662	0.498	model
SalsaNetRec	0.267	0.034	0.478	0.290	model
SalsaNetRecLSTM	0.338	0.075	0.565	0.373	model
SalsaNext	0.258	0.074	0.423	0.277	model
SalsaNextRecLSTM	0.327	0.039	0.548	0.394	model
DDRNet	0.345	0.080	0.768	0.187	model
DDRNetOC	0.323	0.015	0.739	0.216	model
DDRNetDA	0.369	0.040	0.705	0.361	model
Segformer	0.230	0.166	0.182	0.343	model

Models trained on SemanticKITTI + RELLIS-3D

Model	$mIoU$ , %	$IoU$ [human], %	$IoU$ [surface], %	$IoU$ [obstacle], %	Checkpoint
SalsaNet	0.712	0.733	0.760	0.643	model
SalsaNetRec	0.481	0.446	0.590	0.406	model
SalsaNetRecLSTM	0.689	0.770	0.761	0.537	model
SalsaNext	0.663	0.799	0.684	0.505	model
SalsaNextRecLSTM	0.736	0.813	0.784	0.612	model
DDRNet	0.580	0.697	0.535	0.509	model
DDRNetOC	0.616	0.724	0.586	0.539	model
DDRNetDA	0.541	0.614	0.507	0.503	model
Segformer	0.290	0.412	0.054	0.404	model

Models trained on DAPS-1

Model	$mIoU$ , %	$IoU$ [human], %	$IoU$ [surface], %	$IoU$ [obstacle], %	Checkpoint
SalsaNet	0.575	0.479	0.741	0.504	model
SalsaNetRec	0.609	0.596	0.748	0.482	model
SalsaNetRecLSTM	0.624	0.532	0.783	0.557	model
SalsaNext	0.643	0.706	0.620	0.602	model
SalsaNextRecLSTM	0.759	0.709	0.746	0.823	model
DDRNet	0.516	0.354	0.737	0.458	model
DDRNetOC	0.562	0.403	0.705	0.577	model
DDRNetDA	0.531	0.360	0.718	0.515	model
Segformer	0.393	0.355	0.418	0.407	model