FundusExpert

This repository is the official implementation of the paper Constructing Ophthalmic MLLM for Positioning-diagnosis Collaboration Through Clinical Cognitive Chain Reasoning.

🌐 Conference | 📝 Publication | 📖 arXiv

🤗 Model | 🤗 Dataset

Introduction

This paper introduces FundusExpert, an ophthalmology-specific MLLM with integrated positioning-diagnosis reasoning capabilities, along with FundusGen, a dataset constructed through the intelligent Fundus-Engine system.

Data Processing

📌 Data Processing Doc

Setup

Clone this repository and install the dependencies.

Please refer to InternVL Installation or use the src/internvl25_requirements.txt to build the environment.

Quick Start

Inference with single GPU:

python src/quick_start.py

Evaluation

📌 Evaluation Doc

Acknowledgements

Our model is based on OpenGVLab/InternVL. Our evaluation code is based on open-compass/VLMEvalKit. Our segmentation model and code come from MIC-DKFZ/nnUNet. We would like to thank their excellent work and open source contributions.

Citation

@inproceedings{liu2025constructing,
  title={Constructing Ophthalmic MLLM for Positioning-diagnosis Collaboration Through Clinical Cognitive Chain Reasoning},
  author={Liu, Xinyao and Song, Diping},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={21547--21556},
  year={2025}
}