Spectral Heat Flow for Conservative Token Condensation in Vision-Language Models

Contents

• Installation
• Dataset
• Evaluation
• Implementation
• Citation
• Acknowledgement

Installation

1. Clone this repository.

git clone https://github.com/Lzy-dot/SpecFlow.git
cd SpecFlow

2. Prepare the environment.

Follow the official LLaVA installation instructions, or create an equivalent environment:

conda create -n specflow python=3.10 -y
conda activate specflow
pip install --upgrade pip

Then install the dependencies following the official LLaVA setup.

Dataset

We follow the benchmark layout used by LLaVA. Please prepare evaluation datasets under playground/data/eval/ with the following structure:

SpecFlow/
└── playground/
    └── data/
        └── eval/
            ├── gqa/
            ├── mmbench/
            ├── mmbench_cn/
            ├── MME/
            ├── pope/
            ├── scienceqa/
            ├── textvqa/
            ├── vizwiz/
            └── vqav2/

Evaluation

Evaluation scripts are under:

scripts/v1_5/eval/

Run the corresponding script after preparing the benchmark data.

Citation

If you find SpecFlow useful, please consider giving our repository a star (⭐) and citing our paper :)

@inproceedings{li2026spectral,
  title={Spectral Heat Flow for Conservative Token Condensation in Vision-Language Models},
  author={Li, Zhaoyang and Li, Yanjun and Li, Wangkai and Chen, Yujia and Zhang, Tianzhu},
  booktitle={Forty-Third International Conference on Machine Learning},
  year={2026}
}

Acknowledgement

We sincerely appreciate the excellent open-source contributions of LLaVA and Video-LLaVA. We also acknowledge FastV, SparseVLM, HoloV, and other related efforts for their valuable inspiration and insights.