Overview

INSTINCT is a collaborative 3D object detection framework for autonomous driving that achieves state-of-the-art accuracy while requiring only 1/281 to 1/264 of the communication bandwidth compared to existing methods. It introduces instance-level query-based interaction between cooperative agents (vehicles and infrastructure), overcoming the limitations of single-vehicle perception in long-range detection and occlusion scenarios.

News

• [2025.07] INSTINCT is accepted by ICCV 2025.
• [2025.05] Code and configs are released.

Key Contributions

INSTINCT features three core components:

1. Quality-Aware Filtering (QAF) — A learned filtering mechanism that selects high-quality instance features before transmission, suppressing noisy or redundant queries to reduce bandwidth and improve robustness.

2. Dual-Branch Detection Routing (DDR) — Decouples collaboration-irrelevant instances (detectable by a single agent) from collaboration-relevant instances (requiring cross-agent information), routing them through separate detection pathways for more effective collaboration.

3. Cross-Agent Local Instance Fusion (CALIF) — Aggregates local hybrid instance features from multiple agents using two sub-modules:

   • Cross-Domain Adaption (CDA): Aligns feature distributions between ego and cooperative agents.
   • Gaussian Distance Attention (GDA): Models spatial relationships with Gaussian-weighted distance attention for geometrically-aware fusion.

Additionally, an enhanced GT Sampling technique is introduced to facilitate training with diverse hybrid instance features.

Main Results

Comparison with State-of-the-Art

INSTINCT achieves 13.23% relative improvement on DAIR-V2X (AP₇₀) and 33.08% relative improvement on V2V4Real (AP₅₀) over the previous best intermediate fusion method (CoAlign), while requiring ~10× less communication bandwidth.

Installation

Prerequisites

• Python >= 3.7
• PyTorch >= 1.12
• CUDA >= 11.6
• spconv >= 2.x

Step 1: Clone the Repository

git clone https://github.com/CrazyShout/INSTINCT.git
cd INSTINCT

Step 2: Create Conda Environment

conda create -n instinct python=3.7.16
conda activate instinct

Step 3: Install PyTorch and spconv

# PyTorch 1.12 + CUDA 11.6
pip install torch==1.12.0 torchvision==0.13.0 torchaudio==0.12.0 --index-url https://download.pytorch.org/whl/cu116

# spconv
pip install spconv-cu116

Step 4: Install Dependencies

pip install -r requirements.txt

Step 5: Build CUDA Extensions

# Install the opencood package
python setup.py develop

# Build CUDA kernels (GPU required)
python opencood/utils/setup.py build_ext --inplace
python opencood/pcdet_utils/setup.py build_ext --inplace

Build Troubleshooting: If you encounter THC/THC.h errors during pcdet_utils compilation (common with PyTorch >= 2.0), comment out the following lines in the .cpp files under opencood/pcdet_utils/pointnet2/, roiaware_pool3d/, and iou3d_nms/:

// #include <THC/THC.h>
// extern THCState *state;

Step 6: Install OpenPCDet

cd ..
git clone https://github.com/open-mmlab/OpenPCDet.git
cd OpenPCDet
python setup.py develop
cd ../INSTINCT

conda create -n instinct python=3.10
conda activate instinct

# PyTorch nightly for sm_120 support
pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

# spconv
pip install spconv-cu120

# Use compatible numba (0.48.0 is incompatible with Python 3.10)
pip install numba
pip install --no-deps -r requirements.txt

Data Preparation

The data preparation follows the same procedure as CoAlign and OpenCOOD.

For the DAIR-V2X dataset, please use the supplemented annotations provided by CoAlign.

Getting Started

Training

python opencood/tools/train_simple.py -y dairv2x opencood/hypes_yaml/dairv2x/lidar_only_with_noise/second_CQCPInstance_onecycle.yaml

The -y flag specifies the dataset name. Available options: dairv2x, opv2v, v2v4real, v2xsim, v2xset.

Configs are organized by dataset under opencood/hypes_yaml/<dataset>/.

Inference

python opencood/tools/inference_simple.py --model_dir opencood/logs/<experiment_dir>

Checkpoints

Pre-trained model weights will be released soon. Stay tuned!

Code Structure

INSTINCT/
├── opencood/
│   ├── models/
│   │   ├── second_boxattention_cqcp.py      # Main model entry
│   │   ├── comm_modules/
│   │   │   └── CQCP_head_instance.py        # INSTINCT detection head (QAF + DDR + CALIF)
│   │   └── sub_modules/
│   │       ├── CQCP_transformer.py          # Encoder-decoder transformer
│   │       └── matcher.py                   # Hungarian matching
│   ├── data_utils/datasets/
│   │   └── intermediate_v2_fusion_dataset.py # Data pipeline for INSTINCT
│   ├── hypes_yaml/                          # YAML configs per dataset
│   ├── pcdet_utils/                         # Custom CUDA kernels
│   ├── tools/                               # Training & inference scripts
│   └── visualization/                       # Visualization utilities
├── images/                                  # Demo and architecture figures
└── setup.py

Citation

If you find INSTINCT useful for your research, please consider citing:

@inproceedings{xu2025instinct,
  title={INSTINCT: Instance-Level Interaction Architecture for Query-Based Collaborative Perception},
  author={Xu, Yunjiang and Li, Lingzhi and Wang, Jin and Ouyang, Yupeng and Yang, Benyuan},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={25464--25473},
  year={2025}
}

Acknowledgements

This project is built upon the excellent collaborative perception codebases:

• OpenCOOD
• CoAlign
• ConQueR
• OpenPCDet