README_data.md

Data preprocess

nuScenes

1. Prepare the nuScenes data.

• Download the nuScenes data and follow MotionNet to process the data, we want the data to be saved like this:

  |-- motionnet-data (the processed input data from MotionNet)
  |-- nuScenes-data (downloaded raw data)
  |   |-- maps
  |   |-- samples
  |   |-- sweeps
  |   |-- v1.0-trainval
  

2. Prepare the Foreground/Background data for weak supervision:

• Download the nuScenes-lidarseg data, and extract the lidarseg and v1.0-* folders into nuScenes-data/.

• Then run:

  cd gen_data_pp
  python gen_back_BEV_raw_point.py \
    --root /path_to/nuScenes/nuScenes-data/ \
    --split train \
    --save_back_path /path_to/nuScenes/motionnet-data-back/ \
    --save_rawPC_path /path_to/nuScenes/raw-pc/ \
    --save_info_path /path_to/nuScenes/weak-data-info/ \
    --save_seq_info_path /path_to/nuScenes/scene-info/
  

  Argument	Description
  --root	nuScenes dataset root
  --split	Scene split to process; use train
  --save_back_path	Backward BEV voxel sequences
  --save_rawPC_path	Raw LiDAR scans, lidarseg labels, and pose matrices
  --save_info_path	Weak-supervision metadata per training sequence (tokens and timestamps)
  --save_seq_info_path	Per-scene index file mapping sequences to sample tokens

• Run gen_label_for_nuScenes.py to convert lidarseg semantic labels into per-point FG/BG labels for partial annotation during weak-supervision training:

  cd gen_data_pp
  python gen_label_for_nuScenes.py \
    --raw_pc_root /path_to/nuScenes/raw-pc/ \
    --label_info_root /path_to/nuScenes/raw-pc-label/
  

  Argument	Description
  --raw_pc_root	Input from the previous step (raw-pc/)
  --label_info_root	Output FG/BG labels

3. Prepare the ground / non-ground data for weak and self-supervision:

• Run gen_ground_point_for_nuScenes.py to estimate per-scene ground plane models (used for weak and self-supervision):

  cd gen_data_pp
  python gen_ground_point_for_nuScenes.py \
    --raw_pc_root /path_to/nuScenes/raw-pc/ \
    --weak_info_root /path_to/nuScenes/weak-data-info/ \
    --ground_info_root /path_to/nuScenes/ground-data-info/
  

  Argument	Description
  --raw_pc_root	Raw point clouds from step 2 (raw-pc/)
  --weak_info_root	Weak-supervision sequence metadata (weak-data-info/)
  --ground_info_root	Output ground plane models (ground-data-info/)

  The directory layout after all nuScenes preprocessing steps:

  |-- motionnet-data
  |-- nuScenes-data
  |   |-- lidarseg
  |   |-- maps
  |   |-- samples
  |   |-- sweeps
  |   |-- v1.0-trainval
  |-- motionnet-data-back/
  |-- raw-pc/
  |-- raw-pc-label/
  |-- weak-data-info/
  |-- scene-info/
  |-- ground-data-info/
  

Waymo

1. Prepare the Waymo data:

• Download the Waymo Open Dataset (v1.3.2) and install pip install waymo-open-dataset-tf-2-6-0.

• Run step1_waymo_prepare.py to convert .tfrecord files into per-frame .npy point clouds and per-scene .pkl annotations. Run it once per split (training and validation) so that step 2 can consume both subfolders:

  cd gen_data_pp
  python step1_waymo_prepare.py \
    --ROOT_DIR /path_to/Waymo-tfrecord/ \
    --SAVE_ROOT_DIR /path_to/Waymo/Waymo-npy-data/ \
    --split training
  
  python step1_waymo_prepare.py \
    --ROOT_DIR /path_to/Waymo-tfrecord/ \
    --SAVE_ROOT_DIR /path_to/Waymo/Waymo-npy-data/ \
    --split validation
  

  Argument	Description
  --ROOT_DIR	Parent directory containing the downloaded Waymo .tfrecord splits (expects training/ / validation/)
  --SAVE_ROOT_DIR	Parent output directory; results are written to <SAVE_ROOT_DIR>/<split>/
  --split	Which split to convert; one of training, validation

2. Prepare the input data, motion ground truth, and Foreground/Background data for weak supervision:

• Run step2_motionnet_waymo_generate_weak.py to build BEV input voxels, motion ground truth, backward BEV voxels, and weak-supervision metadata for both train and val splits:

  cd gen_data_pp
  python step2_motionnet_waymo_generate_weak.py \
    --ROOT_DIR /path_to/Waymo/Waymo-npy-data/ \
    --SAVE_ROOT_DIR /path_to/Waymo/
  

  Argument	Description
  --ROOT_DIR	Waymo .npy data root produced by step1_waymo_prepare.py (with training/, validation/, and ImageSets/)
  --SAVE_ROOT_DIR	Output root for motionnet-data/, motionnet-data-back/, raw-pc/, weak-data-info/, and scene-info/

• Run gen_label_for_waymo.py to convert per-point semantic labels into FG/BG labels for partial annotation during weak-supervision training:

  cd gen_data_pp
  python gen_label_for_waymo.py \
    --raw_pc_root /path_to/Waymo/raw-pc/ \
    --label_info_root /path_to/Waymo/raw-pc-label/
  

  Argument	Description
  --raw_pc_root	Raw point clouds from the previous step (raw-pc/)
  --label_info_root	Output FG/BG labels

3. Prepare the ground / non-ground data for weak and self-supervision:

• Run gen_ground_point_for_waymo.py to estimate per-scene ground plane models (used for weak and self-supervision):

  cd gen_data_pp
  python gen_ground_point_for_waymo.py \
    --raw_pc_root /path_to/Waymo/raw-pc/ \
    --weak_info_root /path_to/Waymo/weak-data-info/ \
    --ground_info_root /path_to/Waymo/ground-data-info/
  

  Argument	Description
  --raw_pc_root	Raw point clouds from step 2 (raw-pc/)
  --weak_info_root	Weak-supervision sequence metadata (weak-data-info/)
  --ground_info_root	Output ground plane models (ground-data-info/)

  The directory layout after all Waymo preprocessing steps:

  |-- Waymo-npy-data/ (converted from .tfrecord by step1)
  |   |-- training/
  |   |-- validation/
  |   |-- ImageSets/
  |-- motionnet-data/
  |   |-- train/
  |   |-- val/
  |-- motionnet-data-back/
  |-- raw-pc/
  |-- raw-pc-label/
  |-- weak-data-info/
  |-- scene-info/
  |-- ground-data-info/
  

Acknowledgement

The data generation references the codes in the following repos.

• MotionNet
• nuScenes
• Waymo Open Dataset
• OpenPCDet