R4Det

Paper | Zhongyu Xia, Yousen Tang, Yongtao Wang, Zhifeng Wang, Weijun Qin

This repository contains the official PyTorch implementation of: R4Det: 4D Radar-Camera Fusion for High-Performance 3D Object Detection

Introduction

R4Det is a unified radar-camera fusion framework for robust 3D object detection. Our framework contains three key components:

• Panoramic Depth Fusion (PDF) Enhances geometric depth estimation through joint absolute-relative depth modeling.
• Deformable Gated Temporal Fusion (DGTF) Performs pose-free temporal alignment and gated temporal feature aggregation.
• Instance-Guided Dynamic Refinement (IGDR) Uses 2D instance-aware prototypes to dynamically calibrate and purify BEV representations.

Qualitative comparisons on challenging driving scenarios shows that R4Det achieves more robust object localization and better small-object detection under sparse radar observations and low-light environments.

Main Results

Experiments on both TJ4DRadSet and View-of-Delft (VoD) demonstrate the effectiveness of the proposed framework.

Getting Started

KnowVal, HENet, R4Det, RCBEVDet, and TEOcc were developed under the same framework. You can easily merge these repositories into one. If you have prepared the environment for any of them, you do not need to create a new environment.

Environment Setup

The project relies on customized CUDA operators and OpenMMLab-based 3D perception modules.

Please refer to Environment Setup Guide for detailed installation instructions.

Dataset Preparation

Please refer to Dataset Preparation Guide for dataset organization and preprocessing steps.

Training

Distributed training:

bash ./tools/dist_train.sh ${CONFIG_FILE} ${GPU_NUM}

Example:

bash ./tools/dist_train.sh \
./configs/sgdet3d/vod-R4Det_det3d_2x4_12e.py 1

You may replace the configuration file with your own experiment configuration.

Evaluation

Evaluate on TJ4DRadSet

bash test_TJ4D.sh

Please modify the checkpoint path and configuration inside the test scripts before evaluation if necessary.

Model Zoo

Dataset	Model	Weights	Config
TJ4DRadSet	Pretrained	Hugging Face	configs/r4det/TJ4D-R4Det_pretrain_2x4_12e.py
TJ4DRadSet	R4Det	Hugging Face	configs/r4det/TJ4D-R4Det_det3d_2x4_12e.py

We recommend placing checkpoints under:

checkpoints/

Acknowledgements

We sincerely thank the authors of the following open-source projects:

• SGDet3D
• MMDetection3D
• Detectron2

Citation

If you find this work useful for your research, please consider citing:

@article{xia2026r4det,
  title={R4Det: 4D Radar-Camera Fusion for High-Performance 3D Object Detection},
  author={Xia, Zhongyu and Tang, Yousen and Wang, Yongtao and Wang, Zhifeng and Qin, Weijun},
  journal={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2026}
}

License

This project is released for academic research purposes only.

For commercial licensing requests, please contact wyt@pku.edu.cn.