Pruning All-Rounder: Rethinking and Improving Inference Efficiency for Large Vision Language Models [ICCV' 25]

Although Large Vision-Language Models (LVLMs) have achieved impressive results, their high computational costs pose a significant barrier to wide application. To enhance inference efficiency, most existing approaches can be categorized as parameter-dependent or token-dependent strategies to reduce computational demands. However, parameter-dependent methods require retraining LVLMs to recover performance while token-dependent strategies struggle to consistently select the most relevant tokens. In this paper, we systematically analyze the above challenges and provide a series of valuable insights for inference acceleration. Based on these findings, we propose a novel framework, the Pruning All-Rounder (PAR). Different from previous works, PAR develops a meta-router to adaptively organize pruning flows across both tokens and layers. With a self-supervised learning manner, our method achieves a superior balance between performance and efficiency. Notably, PAR is highly flexible, offering multiple pruning versions to address a range of acceleration scenarios.

Motivation

Method

Quantitative Results

Qualitative Results

Installation

Please see details in LLaVA and Qwen-VL directory.

Acknowlegdements

This codebase is based on LLaVA, Qwen-VL and FastV. Many thanks to the authors for generously sharing their codes!