README.md

April 7, 2026 · View on GitHub

CFMW: Cross-modality Fusion Mamba for Robust Object Detection under Adverse Weather

Haoyuan Li, Qi Hu, Binjia Zhou, You Yao, Jiacheng Lin, Kailun Yang*, Peng Chen*

*Corresponding Authors

🏠 About

In this work, we propose the Cross-modality Fusion Mamba with Weather-removal (CFMW) to augment stability and cost-effectiveness under adverse weather conditions. Leveraging the proposed Perturbation-Adaptive Diffusion Model (PADM) and Cross-modality Fusion Mamba (CFM) modules, CFMW is able to reconstruct visual features affected by adverse weather, enriching the representation of image details. To bridge the gap in relevant datasets, we construct a new Severe Weather Visible-Infrared (SWVI) dataset, encompassing diverse adverse weather scenarios such as rain, haze, and snow.

🔍 Overview

Model

Dataset

Data Preparation

  1. Download the four compressed files here. They require about 20GB of storage space.
  2. Run the following command to merge the two files into one and uncompress it. This will produce a folder named SWVI containing 60K visible images, infrared images and visible images with weather-influence.
cat SWVI.* > SWVI.zip
unzip SWVI.zip

Data Structure

SWVI/
├── README.md
└── infrared
       ├── 03205.png
       ├── 03206.png
       ├── 03207.png
       └── ...
└── visible
       ├── 03205.png
       ├── 03206.png
       ├── 03207.png
       └── ...
└── visible_with_weather
       ├── 03205.png
       ├── 03206.png
       ├── 03207.png
       └── ...
└── labels
       ├── 03205.txt
       ├── 03206.txt
       ├── 03207.txt
       └── ...
├── train.txt
└── val.txt

Environment Preparation

conda create -n your_env_name python=3.8 -y

pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 --index-url https://download.pytorch.org/whl/cu118

pip install -r requirements.txt

pip install -e causal_conv1d>=1.1.0

pip install -e mamba-1p1p1

Run

PADM Pre-processing

Training

cd PADM/
export CUDA_VISIBLE_DEVICES=${CUDA_VISIBLE_DEVICES}
python train_diffusion.py

Inference

cd PADM/
python eval_diffusion.py

CFMW Detection

Training

export CUDA_VISIBLE_DEVICES=${CUDA_VISIBLE_DEVICES}
python train.py

Inference

python detect_twostram.py

Demo

Evaluation

python test.py

🔗 Citation

If you find our work and this codebase helpful, please consider starring this repo 🌟 and cite:

@ARTICLE{11077409,
  author={Li, Haoyuan and Hu, Qi and Zhou, Binjia and Yao, You and Lin, Jiacheng and Yang, Kailun and Chen, Peng},
  journal={IEEE Transactions on Circuits and Systems for Video Technology}, 
  title={CFMW: Cross-Modality Fusion Mamba for Robust Object Detection Under Adverse Weather}, 
  year={2025},
  volume={35},
  number={12},
  pages={12066-12081},
  doi={10.1109/TCSVT.2025.3587918}}

👏 Acknowledgements