U2Fusion-PyTorch

U2Fusion: A Unified Unsupervised Image Fusion Network（TPAMI 2022）

News

• The inference code and environment configuration have been released.
• The results of every task have been provided.

---

Overview

This study proposes a novel unified and unsupervised end-to-end image fusion network, termed as U2Fusion, which is capable of solving different fusion problems, including multi-modal, multi-exposure, and multi-focus cases. Now, with the upadta to the code frameworl, I will provide the version migrated to PyTorch. The current repository supports quick testing with pretrained checkpoints.

Tips:

Large files should be downloaded separately, including the following files:

For training:

• Training dataset and vgg16.npy
  

---

Environment and Inference

The code was tested with the following main environment:

Python == 3.10.20  PyTorch == 2.8.0 +cu126  TorchVision == 0.23.0 +cu126   

More detailed package versions are listed in requirements.txt

---

Visual Results

Results on TNO

Results on RoadScene

Results on Medical

Results on Multi-Exposure

Results on Multi-Focus

---

Repository Structure

U2Fusion-PyTorch/
├── checkpoints/
│   └── task3/                   # Final Path
├── test_imgs/
│   ├── vis-ir/
│       ├── TNO/
│           ├── vir/             # Visible Images
│           └── ir/              # Infrared Images
│       └── RoadScene/
│           ├── vir/             # Visible Images
│           └── ir/              # Infrared Images
│   ├── medical/
│       ├── mri/                 # Magnetic Resonance Images
│       └── pet/                 # Positron Emission Tomography Images
│   ├── multi-exposure/                      
│       ├── dataset1/
│           ├── ue/              # Under-Exposed Images
│           └── oe/              # Over-Exposed Images
│       └── dataset2/
│           ├── ue/              # Under-Exposed Images
│           └── oe/              # Over-Exposed Images
│   └── multi-focus/  
│        ├── far/                # Far-Focused Images    
│        └── near/               # Near-Focused Images
├── img_RGB/                     # RGB Input
├── results/                     # Inference results
├── convert_vgg.py
├── vgg16.py  
├── dataset.py
├── generator.py
├── losses.py
├── model.py
├── train.py
├── test.py                     
├── color.py                     # Its function has been achieved in test
├── requirements.txt
└── README.md

---

Citation

If you find this work useful for your research, please cite our paper:

@article{xu2020u2fusion,
  title={U2Fusion: A Unified Unsupervised Image Fusion Network},
  author={Xu, Han and Ma, Jiayi and Jiang, Junjun and Guo, Xiaojie and Ling, Haibin},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2022},
  publisher={IEEE}
}