Variable Augmented Network for Invertible Modality Synthesis-Fusion

May 30, 2025 · View on GitHub

The Code is created based on the method described in the following paper:
Variable Augmented Network for Invertible Modality Synthesis and Fusion
Y. Wang, R. Liu, Z. Li, S. Wang, C. Yang, Q. Liu
IEEE Journal of Biomedical and Health Informatics
Page: 2898 - 2909, Volume: 27 Issue: 6, 2023.
https://ieeexplore.ieee.org/abstract/document/10070774
Date : Sep. 1, 2021
Version : 1.0
The code and the algorithm are for non-comercial use only.
Copyright 2021, Department of Electronic Information Engineering, Nanchang University.

Optional parameters:

weight: Weight for forward loss.
epoch: Specifies number of iterations.

Visual illustration of the invertible medical image synthesis and fusion in variable augmentation manner

The training pipeline of iVAN

Two visualization results of synthesizing from T1 to T2

Three fusion results of T2-weighted MR and CT images

Train

Prepare your own datasets for VAN

You need to create at least two modality medical images from domain A /data/A and from domain B /data/B. Then you can train the model with the dataset flag --root1 './data/A' --root2 './data/B'. Optionally, you can create hold-out test datasets at ./data/A_test and ./data/B_test to test your model.

1to1

python train.py --task=1to1 --out_path="./exps/"

many to 1

python train.py --task=2to1 --out_path="./exps/"

resume training:

To fine-tune a pre-trained model, or resume the previous training, use the --resume flag

Test

python test.py --task=2to1 --out_path="./exps/" --ckpt="./exps/2to1/checkpoint/latest.pth"

python test.py --task=1to1 --out_path="./exps/" --ckpt="./exps/1to1/checkpoint/latest.pth"

Acknowledgement

The code is based on yzxing87/Invertible-ISP

Some examples of invertible and variable augmented network: IVNAC, VAN-ICC, iVAN and DTS-INN.

  • Variable augmentation network for invertible MR coil compression [Paper] [Code]

  • Variable Augmented Network for Invertible Decolorization (基于辅助变量增强的可逆彩色图像灰度化) [Paper] [Code]

  • Virtual coil augmentation for MR coil extrapoltion via deep learning [Paper] [Code]

  • Synthetic CT Generation via Variant Invertible Network for Brain PET Attenuation Correction [Paper] [Code]

  • Temporal Image Sequence Separation in Dual-tracer Dynamic PET with an Invertible Network [Paper] [Code]

  • Invertible and Variable Augmented Network for Pretreatment Patient-Specific Quality Assurance Dose Prediction [Paper]

  • Variable augmented neural network for decolorization and multi-exposure fusion [Paper] [Code] [Slide]

  • Spatial-Temporal Guided Diffusion Transformer Probabilistic Model for Delayed Scan PET Image Prediction [Paper] [Code]