README.md

February 21, 2026 · View on GitHub

✂️ Mixing Importance with Diversity: Joint Optimization for KV Cache Compression in Large Vision-Language Models

Xuyang Liu1,2*, Xiyan Gui3*, Yuchao Zhang1, Linfeng Zhang1✉

1EPIC Lab, Shanghai Jiao Tong University, 2Sichuan University, 3Huazhong University of Science and Technology

💡 The first to identify heterogeneous head-wise redundancy in the KV cache of both LVLMs and LLMs.

🔥 News

  • 2026.01.26 🎊🎊 Our MixKV has been accepted by ICLR 2026!
  • 2025.10.24 🤗🤗 We release our latest work MixKV, a novel KV cache compression method of LVLMs. Code is available!

📌 Highlights

  • Model Compatibility: Works with diverse LVLMs and LLMs (e.g., LLaVA, Qwen-VL, InternVL, Llama, Mistral).
  • Seamless Integration: Plugs into existing KV cache compressors (e.g., SnapKV, AdaKV, SparseMM).
  • Universal Improvements: Boosts baseline compressor performance across models and tasks.
  • High Efficiency: Delivers gains without compromising speed or memory efficiency.

🛠 Preparation

  1. Clone this repository:
git clone https://github.com/xuyang-liu16/MixKV.git
cd MixKV
  1. Init your environment
conda create -n mixkv python=3.10 -y
conda activate mixkv
  1. Install packages

Compile CUDA code for Flatten Cache Storage. If you encounter a CUDA compile error, please check your GPU Virtual Architecture and GPU Feature, then change the corresponding compile flag in csrc/build.py

pip install packaging torch==2.5.1
pip uninstall ninja && pip cache purge && pip install ninja --no-cache-dir
cd csrc && make
cd ..

Install other packages

pip install -e .
pip install flash-attn==2.4.1 --no-build-isolation # currently only support FlashAttention
pip install qwen-vl-utils
  1. Install lmms-eval for evaluation

We use the lmms-eval toolkit to evaluate all methods and models. Please install lmms-eval:

cd lmms-eval
pip install -e .

🚀 Performance Evaluation

To evaluate LLaVA-NeXT-Mistral-7B, InternVL3-8B, and Qwen2-VL-7B, you can use:

bash scripts/eval/mistral.sh
bash scripts/eval/internvl2.sh
bash scripts/eval/qwen.sh

⚡ Efficiency Analysis

To evaluate the latency and peak memory by different settings, you can use:

bash scripts/others/speed_and_memory.sh

📌 Citation

Please consider citing our paper in your publications, if our findings help your research.

@article{liu2025mixkv,
  title={Mixing Importance with Diversity: Joint Optimization for KV Cache Compression in Large Vision-Language Models},
  author={Liu, Xuyang and Gui, Xiyan and Zhang, Yuchao and Zhang, Linfeng},
  journal={arXiv preprint arXiv:2510.20707},
  year={2025}
}

👍 Acknowledgment

Our codebase is conducted on SparseMM and we extend our gratitude to the open-source efforts.