🎈 News

• [2025.6.25] Our work has been accepted by ICCV 2025！
• [2025.7.24] Our work is selected as the highlight paper！

📑 TODO

• Release the complete code

⭐ Abstract

In recent years, it has been found that “grandmother cells” in the primary visual cortex (V1) of macaques can directly recognize visual input with complex shapes. This inspires us to examine the value of these cells in promoting the research of medical image segmentation. In this paper, we design a Similarity Memory Prior Network (Sim-MPNet) for medical image segmentation. Specifically, we propose a Dynamic Memory Weights–Loss Attention (DMW-LA), which matches and remembers the category features of specific lesions or organs in medical images through the similarity memory prior in the prototype memory bank, thus helping the network to learn subtle texture changes between categories. DMW-LA also dynamically updates the similarity memory prior in reverse through Weight-Loss Dynamic (W-LD) update strategy, effectively assisting the network directly extract category features. In addition, we propose the Double-Similarity Global Internal Enhancement Module (DS-GIM) to deeply explore the internal differences in the feature distribution of input data through cosine similarity and euclidean distance. Extensive experiments on four public datasets show that Sim-MPNet has better segmentation performance than other state-of-the-art methods.

🚀 Introduction

• Motivation. (a) Schematic diagram of the primate visual system. (b) We use similarity memory priors stored in the prototype memory bank to imitate “grandmother cells” in V1, and utilize these features to directly match various organs in medical images.

• Compared with the existing works. DMW-LA uses similarity memory priors to directly identify and extract category features of organs.

📻 Overview

• Overview of Sim-MPNet. The network consists of two encoders, one decoder and one skip connection, and it efficiently models intra-class and global dependencies of input features through the dual encoder structure.

🎮 Getting Started

1. Install Environment

torch==2.6.0
numpy==1.24.4
timm==1.0.15
scipy==1.15.3
einops==0.8.1
faiss-gpu==1.7.2
scikit-learn==1.7.0
tensorboardX==2.6.4
medpy==0.5.2
seaborn==0.13.2
segmentation_mask_overlay==0.4.4
thop==0.1.1.post2209072238
h5py==3.14.0
torchsummaryX==1.3.0
imgaug==0.4.0

2. Prepare Datasets

• Download datasets: Synapse from this link, SegPC-2021 from this link, ACDC from this link, ISIC-2018 from this link.

• Folder organization: put datasets into ./data folder.

3. Pretrained Model

• You should download the pretrained MaxViT models from link, and then put it in the './pretrained_pth/maxvit/' folder for initialization.

4. Train the Sim-MPNet

cd into Sim-MPNet

If you want to train on the Synapse dataset, please modify line 62 of networks.py with num_k=9 and feature_dim=256 If you want to train on the ACDC dataset, please modify line 62 of networks.py with num_k=4 and feature_dim=128 If you want to train on the SegPC-2021 dataset, please modify line 62 of networks.py with num_k=3 and feature_dim=256 If you want to train on the ISIC-2018 dataset, please modify line 62 of networks.py with num_k=2 and feature_dim=256

Training run CUDA_VISIBLE_DEVICES=0 python -W ignore train_synapse.py

5. Test the Sim-MPNet

cd into Sim-MPNet 

Testing run CUDA_VISIBLE_DEVICES=0 python -W ignore test_synapse.py

If you want to train and test other datasets, simply modify the corresponding dataset paths and categories in train_stynapse.cy and test_stynapse.cy.

✨ Quantitative comparison

🖼️ Visualization

Citations

@inproceedings{simmpnet,
  title={Similarity Memory Prior is All You Need for Medical Image Segmentation},
  author={Hao Tang, Zhiqing Guo, Liejun Wang and Chao Liu},
  booktitle={International Conference on Computer Vision (ICCV)},
  month={June},
  year={2025}
}