Getting Started for Point Cloud Pre-training

Please download and preprocess the point cloud datasets according to the dataset guidance

Download the labeled and pseudo-labeled data

• Download the labeled data from ONCE dataset

  Note that you can find in once dataset yaml that, we use the once_infos_train_vehicle.pkl and once_infos_val_vehicle.pkl, which are generated by merging the the 'Car', 'Bus', 'Truck' classes into the 'Vehicle' classes.

  You can download the once_infos_train_vehicle.pkl and once_infos_val_vehicle.pkl here: once_infos_val_vehicle and once_infos_train_vehicle

• Download the pseudo-labeled data from ONCE unlabeled dataset

  We pseudo-label the unlabeled data and give the pseudo-labeled results here: once_small_pseudo.pkl, once_medium_pseudo.pkl, and once_large_pseudo.pkl.

How to run AD-PT

• Take pre-training on small pseudo label set as an example:

    cd tools
    sh scripts/PRETRAIN/dist_train_ad-pt.sh ${NUM_GPUS} \
    --cfg_file ./cfgs/once_models/pretrain_models/once_ad-pt_pretrain_small.yaml
  

  or

    cd tools
    sh scripts/PRETRAIN/slurm_train_ad-pt.sh ${PARTITION} ${JOB_NAME} ${NUM_NODES} \ 
    --cfg_file ./cfgs/once_models/pretrain_models/once_ad-pt_pretrain_small.yaml
  

  Note you can choose small / medium / large pseudo set by changing the dataset config file (once_ad-pt_pretrain_small.yaml / once_ad-pt_pretrain_medium.yaml / once_ad-pt_pretrain_large.yaml)

• Fine-tuning on downstream dataset:

    cd tools
    sh scripts/dist_train.sh ${NUM_GPUS} \
    --cfg_file ./cfgs/waymo_models/pv_rcnn_plusplus_resnet.yaml \
    --pretrained_model ${PRETRAINED_CHECKPOINT}
  

  or

    cd tools
    sh scripts/slurm_train.sh ${PARTITION} ${JOB_NAME} ${NUM_NODES} \
    --cfg_file ./cfgs/waymo_models/pv_rcnn_plusplus_resnet.yaml \
    --pretrained_model ${PRETRAINED_CHECKPOINT}
  

  ${PRETRAINED_CHECKPOINT} denotes the pre-trained checkpoints obtained using AD-PT method.

AD-PT pre-trained checkpoints

• For rapid fine-tuning on downstream datasets, we also release the pre-trained checkpoint using our AD-PT

  Pre-training Method	Pre-trained data	Pre-trained model
  AD-PT	ONCE PS-100K	once-100K-ckpt
  AD-PT	ONCE PS-500K	once-500K-ckpt
  AD-PT	ONCE PS-1M	once-1M-ckpt

AD-PT Results:

We report the downstream fine-tuning results using our AD-PT pre-trained backbones.

Fine-tuning Results on Waymo:

	Data amount	Overall	Vehicle	Pedestrian	Cyclist
SECOND (From scratch)	3%	52.00 / 37.70	58.11 / 57.44	51.34 / 27.38	46.57 / 28.28
SECOND (AD-PT)	3%	55.41 / 51.78	60.53 / 59.93	54.91 / 45.78	50.79 / 49.65
SECOND (From scratch)	20%	60.62 / 56.86	64.26 / 63.73	59.72 / 50.38	57.87 / 56.48
SECOND (AD-PT)	20%	61.26 / 57.69	64.54 / 64.00	60.25 / 51.21	59.00 / 57.86
CenterPoint (From scratch)	3%	59.00 / 56.29	57.12 / 56.57	58.66 / 52.44	61.24 / 59.89
CenterPoint (AD-PT)	3%	61.21 / 58.46	60.35 / 59.79	60.57 / 54.02	62.73 / 61.57
CenterPoint (From scratch)	20%	66.47 / 64.01	64.91 / 64.42	66.03 / 60.34	68.49 / 67.28
CenterPoint (AD-PT)	20%	67.17 / 64.65	65.33 / 64.83	67.16 / 61.20	69.39 / 68.25
PV-RCNN++ (From scratch)	3%	63.81 / 61.10	64.42 / 63.93	64.33 / 57.79	62.69 / 61.59
PV-RCNN++ (AD-PT)	3%	68.33 / 65.69	68.17 / 67.70	68.82 / 62.39	68.00 / 67.00
PV-RCNN++ (From scratch)	20%	69.97 / 67.58	69.18 / 68.75	70.88 / 65.21	69.84 / 68.77
PV-RCNN++ (AD-PT)	20%	71.55 / 69.23	70.62 / 70.19	72.36 / 66.82	71.69 / 70.70

Fine-tuning Results on nuScenes:

	Data amount	mAP	NDS	Car	Truck	CV.	Bus	Trailer	Barrier	Motorcycle	Bicycle	Pedestrian	Cyclist
SECOND (From scratch)	5%	29.24	39.74	67.69	33.02	7.15	45.91	17.67	25.23	11.92	0.00	53.00	30.74
SECOND (AD-PT)	5%	37.69	47.95	74.89	41.82	12.05	54.77	28.92	34.41	23.63	3.19	63.61	39.54
SECOND (From scratch)	100%	50.59	62.29	-	-	-	-	-	-	-	-	-	-
SECOND (AD-PT)	100%	52.23	63.04	83.12	52.86	15.24	68.58	37.54	59.48	46.01	20.44	78.96	60.05
CenterPoint (From scratch)	5%	42.68	50.41	77.82	43.61	10.65	44.01	18.71	52.95	36.26	16.76	37.62	54.52
CenterPoint (AD-PT)	5%	44.99	52.99	78.90	43.82	11.13	55.16	21.22	55.10	39.03	17.76	72.28	55.43
CenterPoint (From scratch)	100%	56.2	64.5	84.8	53.9	16.8	67.0	35.9	64.8	55.8	36.4	83.1	63.4
CenterPoint (AD-PT)	100%	57.17	65.48	84.86	54.37	16.09	67.354	36.06	64.31	58.50	40.58	83.53	66.05

Fine-tuning Results on KITTI:

	Data amount	mAP ( Mod.)	Car (mod.)	Pedestrian (Mod.)	Cyclist (Mod.)
SECOND (From scratch)	20%	61.70	78.83	47.23	59.06
SECOND (AD-PT)	20%	65.95	80.70	49.67	67.50
SECOND (From scratch)	100%	66.70	80.78	52.61	66.71
SECOND (AD-PT)	100%	67.58	81.39	53.58	67.78
PV-RCNN (From scratch)	20%	66.71	82.52	53.33	64.28
PV-RCNN (AD-PT)	20%	69.43	82.75	57.59	67.96
PV-RCNN (From scratch)	100%	70.57	84.50	57.06	70.14
PV-RCNN (AD-PT)	100%	73.01	84.75	60.79	73.49