Feature-prompting GBMSeg: One-Shot Reference Guided Training-Free Prompt Engineering for Glomerular Basement Membrane Segmentation

Xueyu Liu, Guangze Shi, Rui Wang, Yexin Lai, Jianan Zhang, Lele Sun, Quan Yang, Yongfei Wu*, Weixia Han, Ming Li, and Wen Zheng
¹Taiyuan University of Technology,   ²The Second Affiliated Hospital of Shanxi Medical University，  ³Shanxi Provincial People's Hospital

🚀🚀This work has been accepted by MICCAI2024!🚀🚀

We present GBMSeg, a training-free framework that automates the segmentation and measurement of the glomerular basement membrane (GBM) in TEM using only one-shot reference images. GBMSeg leverages the robust feature matching capabilities of pretrained foundation models (PFMs) to generate initial prompts, designs novel prompting engineering for optimized prompting methods, and utilizes a class-agnostic segmentation model to obtain the final segmentation result.

Usage

Setup

• Cuda 12.0
• Python 3.9.18
• PyTorch 2.0.0

Datasets

../                          # parent directory
├── ./data                   # data path
│   ├── reference_images      # the one-shot reference image
│   ├── reference_masks       # the one-shot reference mask
│   ├── target_images        # testing images

Usage

python main.py

Citation

If you find this project useful in your research, please consider citing:

@InProceedings{Liu_Featureprompting_MICCAI2024,
        author = { Liu, Xueyu and Shi, Guangze and Wang, Rui and Lai, Yexin and Zhang, Jianan and Sun, Lele and Yang, Quan and Wu, Yongfei and Li, Ming and Han, Weixia and Zheng, Wen},
        title = { { Feature-prompting GBMSeg: One-Shot Reference Guided Training-Free Prompt Engineering for Glomerular Basement Membrane Segmentation } },
        booktitle = {proceedings of Medical Image Computing and Computer Assisted Intervention -- MICCAI 2024},
        year = {2024},
        publisher = {Springer Nature Switzerland},
        volume = {LNCS 15009},
        month = {October},
        page = {276 -- 285}
}

Acknowledgement

Thanks DINOv2, SAM. for serving as building blocks of GBMSeg.