Getting Started & Problem Definition

March 19, 2023 · View on GitHub

The purpose of an Semi-supervised Domain Adaptation (SSDA) task is to learn a generalized model or backbone FF on a labeled source domain set ss and a semi-supervised target domain set tt. For the target domain, a portion of labeled samples and the remaining unlabeled samples are assumed to be available during the whole adaptation process.

 

Getting Started & Task Challenges

The major difference between SSDA and ADA is that, the ADA requires the model to select the most informative unlabeled samples to perform the annotation process, while the SSDA assmues that some labeled samples from the target domain can be obtained in advance. The key point of SSDA is how to leverage the unlabeled data that may present a significant domain gap with the source domain data.

   

Getting Started ## Training & Testing for SSDA setting

Here, we take Waymo-to-nuScenes adaptation as an example.

Pretraining stage: train the source-only model on the labeled source domain:

  • Train FEAT=3 (X,Y,Z) without SN (statistical normalization) using multiple GPUs
sh scripts/dist_train.sh ${NUM_GPUs} \
--cfg_file ./cfgs/SSDA/waymo_nusc/source_only/pvrcnn_feat_3_vehi.yaml
  • Train FEAT=3 (X,Y,Z) without SN (statistical normalization) using multiple machines
sh scripts/slurm_train.sh ${PARTITION} ${JOB_NAME} ${NUM_NODES} \
--cfg_file ./cfgs/SSDA/waymo_nusc/source_only/pvrcnn_feat_3_vehi.yaml
  • Train other baseline detectors such as Second using multiple GPUs
sh scripts/dist_train.sh ${NUM_GPUs} \
--cfg_file ./cfgs/SSDA/waymo_nusc/source_only/second_feat_3_vehi.yaml
  • Train other baseline detectors such as PV-RCNN++ using multiple GPUs
sh scripts/dist_train.sh ${NUM_GPUs} \
--cfg_file ./cfgs/SSDA/waymo_nusc/source_only/pvplus_feat_3_vehi.yaml

Evaluate the source-pretrained model:

  • We report the best model for all epochs on the validation set.

  • Test the source-only models using multiple GPUs

sh scripts/dist_test.sh ${NUM_GPUs} \
--cfg_file ./cfgs/SSDA/waymo_nusc/source_only/pvrcnn_feat_3_vehi.yaml \
--ckpt ${CKPT}
  • Test the source-only models using multiple machines
sh scripts/slurm_test_mgpu.sh ${PARTITION} ${NUM_NODES} \
--cfg_file ./cfgs/SSDA/waymo_nusc/source_only/pvrcnn_feat_3_vehi.yaml \
--ckpt ${CKPT}
  • Test the source-only models of all ckpts using multiple GPUs
sh scripts/dist_test.sh ${NUM_GPUs} \
--cfg_file ./cfgs/SSDA/waymo_nusc/source_only/pvrcnn_feat_3_vehi.yaml \
--eval_all
  • Test the source-only models of all ckpts using multiple machines
sh scripts/slurm_test_mgpu.sh ${PARTITION} ${NUM_NODES} \
--cfg_file ./cfgs/SSDA/waymo_nusc/source_only/pvrcnn_feat_3_vehi.yaml \
--eval_all

Fine-tuning stage: fine-tune the model on the labeled target domain:

  • You need to set the --pretrained_model ${PRETRAINED_MODEL} when finish the pretraining model stage
  • Train FEAT=3 (X,Y,Z) without SN (statistical normalization) using multiple machines
sh scripts/slurm_train.sh ${PARTITION} ${JOB_NAME} ${NUM_NODES} \
--cfg_file ./cfgs/SSDA/waymo_nusc/pvrcnn/pvrcnn_feat_3_vehi_05_finetune.yaml \ --pretrained_model ${PRETRAINED_MODEL}
  • Train FEAT=3 (X,Y,Z) without SN (statistical normalization) using multiple GPUs
sh scripts/dist_train.sh ${NUM_GPUs} \
--cfg_file ./cfgs/SSDA/waymo_nusc/pvrcnn/pvrcnn_feat_3_vehi_05_finetune.yaml \
--pretrained_model ${PRETRAINED_MODEL}

Semi-Supervised stage: training the model on the labeled and unlabeled target domain

sh scripts/SEMI/slurm_train_semi.sh ${PARTITION} ${JOB_NAME} ${NUM_NODES} \
--cfg_file ./cfgs/SSDA/waymo_nusc/pvrcnn/pvrcnn_feat_3_vehi_05_sess.yaml

sh scripts/SEMI/dist_train_semi.sh ${NUM_GPUs} \
--cfg_file ./cfgs/SSDA/waymo_nusc/pvrcnn/pvrcnn_feat_3_vehi_05_sess.yaml
  • Train FEAT=3 (X,Y,Z) for PSEUDO LABEL using multiple machines
sh scripts/SEMI/slurm_train_semi.sh ${PARTITION} ${JOB_NAME} ${NUM_NODES} \
--cfg_file ./cfgs/SSDA/waymo_nusc/pvrcnn/pvrcnn_feat_3_vehi_05_ps.yaml
  • Train FEAT=3 (X,Y,Z) for PSEUDO LABEL using multiple GPUs
sh scripts/SEMI/dist_train_semi.sh ${NUM_GPUs} \
--cfg_file ./cfgs/SSDA/waymo_nusc/pvrcnn/pvrcnn_feat_3_vehi_05_ps.yaml

Evaluating the model on the labeled target domain of validation set:

  • To test all the saved checkpoints of a specific training setting and draw the performance curve on the Tensorboard:
python test_semi.py \
--cfg_file ${CONFIG_FILE} \
--batch_size ${BATCH_SIZE}
  • To test with multiple machines:
sh scripts/SEMI/slurm_test_semi.sh ${PARTITION} ${NUM_NODES} \ 
--cfg_file ${CONFIG_FILE} \
--batch_size ${BATCH_SIZE}

   

All SSDA Results:

We report the cross-dataset adaptation results including Waymo-to-nuScenes, Waymo-to-ONCE.

  • All LiDAR-based models are trained with 4 NVIDIA A100 GPUs and are available for download.
  • The domain adaptation time is measured with 4 NVIDIA A100 GPUs and PyTorch 1.8.1.

SSDA Results for Waymo-to-NuScenes:

  • SSDA under the setting of 5% labeled training frames and 99% unlabeled training frames.
training timeAdaptationCar@R40download
Second~11 hourssource-only(Waymo)27.85 / 16.43-
Second~0.4 hourssecond_5%_FT45.95 / 26.98model-61M
Second~1.8 hourssecond_5%_SESS47.77 / 28.74model-61M
Second~1.7 hourssecond_5%_PS47.72 / 29.37model-61M
PV-RCNN~24 hourssource-only(Waymo)40.31 / 23.32-
PV-RCNN~1.0 hourspvrcnn_5%_FT49.58 / 34.86model-150M
PV-RCNN~5.5 hourspvrcnn_5%_SESS49.92 / 35.28model-150M
PV-RCNN~5.4 hourspvrcnn_5%_PS49.84 / 35.07model-150M
PV-RCNN++~16 hourssource-only(Waymo)31.96 / 19.81-
PV-RCNN++~1.2 hourspvplus_5%_FT49.94 / 34.28model-185M
PV-RCNN++~4.2 hourspvplus_5%_SESS51.14 / 35.25model-185M
PV-RCNN++~3.6 hourspvplus_5%_PS50.84 / 35.39model-185M
  • SSDA under the setting of 1% labeled training frames and 99% unlabeled training frames.
training timeAdaptationCar@R40download
Second~11 hourssource-only(Waymo)27.85 / 16.43-
Second~0.1 hourssecond_1%_FT41.29 / 23.30model-61M
Second~0.5 hourssecond_1%_SESS43.25 / 24.59model-61M
Second~0.5 hourssecond_1%_PS44.08 / 26.12model-61M
PV-RCNN~24 hourssource-only(Waymo)40.31 / 23.32-
PV-RCNN~0.3 hourspvrcnn_1%_FT47.11 / 32.42model-150M
PV-RCNN~1.4 hourspvrcnn_1%_SESS47.43 / 32.75model-150M
PV-RCNN~1.4 hourspvrcnn_1%_PS47.24 / 32.54model-150M
PV-RCNN++~16 hourssource-only(Waymo)31.96 / 19.81-
PV-RCNN++~0.3 hourspvplus_1%_FT47.65 / 31.44model-185M
PV-RCNN++~1.2 hourspvplus_1%_SESS46.95 / 32.60model-185M
PV-RCNN++~1.1 hourspvplus_1%_PS47.51 / 33.28model-185M

 

SSDA Results for Waymo-to-KITTI:

  • SSDA under the setting of 5% labeled training frames and 95% unlabeled training frames.
training timeAdaptationCar@R40download
Second~11 hourssource-only(Waymo)54.77 / 14.57-
Second~0.2 hourssecond_5%_FT79.84 / 60.53model-61M
Second~0.7 hourssecond_5%_SESS82.01 / 66.58model-61M
Second~0.7 hourssecond_5%_PS83.08 / 69.80model-61M
PV-RCNN~24 hourssource-only(Waymo)67.96 / 27.65-
PV-RCNN~0.3 hourspvrcnn_5%_FT85.88 / 79.49model-150M
PV-RCNN~1.0 hourspvrcnn_5%_SESS88.01 / 79.89model-150M
PV-RCNN~1.0 hourspvrcnn_5%_PS88.64 / 82.08model-150M
PV-RCNN++~21 hourssource-only(Waymo)69.86 / 31.24-
PV-RCNN++~0.3 hourspvplus_5%_FT91.12 / 82.21model-185M
PV-RCNN++~0.9 hourspvplus_5%_SESS91.01 / 83.06model-185M
PV-RCNN++~0.9 hourspvplus_5%_PS91.22 / 84.99model-185M
  • SSDA under the setting of 1% labeled training frames and 99% unlabeled training frames.
training timeAdaptationCar@R40download
Second~11 hourssource-only(Waymo)54.77 / 14.57-
Second~0.1 hourssecond_1%_FT77.29 / 54.69model-61M
Second~0.2 hourssecond_1%_SESS81.53 / 62.41model-61M
Second~0.2 hourssecond_1%_PS81.94 / 64.66model-61M
PV-RCNN~24 hourssource-only(Waymo)67.96 / 27.65-
PV-RCNN~0.1 hourspvrcnn_1%_FT86.30 / 76.65model-150M
PV-RCNN~0.3 hourspvrcnn_1%_SESS87.07 / 79.36model-150M
PV-RCNN~0.3 hourspvrcnn_1%_PS90.24 / 81.59model-150M
PV-RCNN++~21 hourssource-only(Waymo)69.86 / 31.24-
PV-RCNN++~0.1 hourspvplus_1%_FT89.70 / 78.94model-185M
PV-RCNN++~0.3 hourspvplus_1%_SESS90.05 / 81.70model-185M
PV-RCNN++~0.3 hourspvplus_1%_PS90.17 / 82.40model-185M