Latest news:

• 08/07/2024: dataset request form is added
• 07/07/2024: create repository

RadVRU dataset

The RadVRU dataset is a collection of synchronized radar and RGB data in urban city environments. It contains 317 sequences of 10 s to 20 s in duration. Each sequence contains radar and RGB data recorded. There are approximately 78,000 frames with the two sensors synchronized. These frames are representative of challenging driving scenarios, including different lighting conditions and complex traffic with cars, pedestrians, and cyclists.

Sensor specifications

The acquisition hardware consists of a 77 GHz FMCW radar (TI AWR1243), a full HD RGB camera (Realsense D435), and a 3D LiDAR (Ouster OS1-128) mounted on the top of a vehicle. The world coordinate system is coupled to the EGO-vehicle and coupled to the Ouster os-1 lidar such that the world center is on the ground plane, directly below the lidar. The world yaw follows the compass heading convention with yaw = 0 in the direction of travel of the car, increasing to the right. Their extrinsic parameters are provided together with the dataset. The sensors’ specifications are detailed in the table below.

Dataset structure

The RadVRU folder contains 25 recorded sequences. Each sequence contains several sub-sequences. Each sub-sequence contains:

• The RAD radar cubes saved in a binary file with resolution [doppler, range, azi] = [128, 128, 16].
• The meta data specifing the range, azimuth, and velocity bins.
• The camera data compressed in MJPEG format.
• Vision feed-forward array generated by projecting YOLOv8 detections to BEV with resolution [range, azi] = [128, 128].

RadVRU
├── realsense_20200916_19h26m02s_2997
│   ├── radar
│   │   ├── 0000
│   │   |   ├── rangebins.txt
│   │   |   ├── azimuthbins.txt
│   │   |   ├── velocitybins.txt
│   │   |   ├── Ramap000000
│   │   |   ├── Ramap000001
│   │   |   ├── ...
│   │   ├── 0001
│   │   ├── ...
│   ├── rgb
│   │   ├── 0000
│   │   |   ├── 000000.jpg
│   │   |   ├── 000001.jpg
│   │   |   ├── ...
│   │   ├── 0001
│   │   ├── ...
│   ├── vf
│   │   ├── 0000
│   │   |   ├── yoloOCCmap_000000.bin
│   │   |   ├── yoloOCCmap_000001.bin
│   │   |   ├── ...
│   │   ├── 0001
│   │   ├── ...
├── ...

Labels

The annotations are performed by using a camera- and a LiDAR-based object detection model to pre-label the instances, and then these labels are manually corrected by human annotators. The pedestrians are annotated as single points on the ground plane. We provide Vulnerable Road User (including pedestrians and cyclists) ground truth for training and evaluation of detection algorithms. The ground truth consists of a frame ID and ground plane position. Labels are stored in a separate pickle file. We also provide the testing sequences used for evaluation.

Download instructions

To download the dataset, please fill out this form to obtain the download link.

Visualization

We provide simple Python scripts to read and visualize the data.

Prerequisites

• Python 3.7
• Pytorch 1.13.1
• CUDA 11.0 or higher

First of all, clone the code:

git clone https://github.com/weiyulee/RadVRU.git

Install all the python dependencies then simply run:

python ./scripts/visualizer.py

Citation

If you find this code useful for your research, please cite our paper:

@InProceedings{Lee_2024_ECCV,
               author = {Lee, Wei-Yu and Dimitrievski, Martin and Van Hamme, David and Aelterman, Jan and Jovanov, Ljubomir and Philips, Wilfried},
               title = {CARB-Net: Camera-Assisted Radar-Based Network for Vulnerable Road User Detection},
               booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)},
               year = {2024},
               }

License

Please see LICENSE.md for more details.