GSTEP: Global Spatio-Temporal Density-Driven Visual Token Pruning for Efficient Video Large Language Models

ACM Multimedia 2026

Official PyTorch implementation of GSTEP, a training-free visual-token pruning method for Video Large Language Models. GSTEP currently supports:

• LLaVA-OneVision-7B
• Qwen2.5-VL-7B-Instruct
• Qwen3-VL-8B-Instruct

GSTEP models a video as a continuous spatio-temporal information flow rather than a collection of independently compressed segments. At 75% visual-token pruning on LLaVA-OneVision-7B, GSTEP preserves up to 100.2% of the original average benchmark performance and achieves a 1.17x end-to-end speedup.

✨ Highlights

• Training-free: requires no retraining or task-specific fine-tuning.
• Global selection: allocates one token budget over the complete video without hard scene or segment boundaries.
• Spatio-temporal density: combines continuous temporal change with intra-frame spatial saliency.
• Coverage-aware pruning: density-guided farthest-point sampling balances information density and global feature-space coverage.
• Architecture support: provides integrations for LLaVA-OneVision, Qwen2.5-VL, and Qwen3-VL.

🧩 Method

GSTEP consists of three stages:

1. Continuous temporal density. A centered frame-level semantic-change curve is stabilized using one-dimensional Gaussian smoothing.
2. Intra-frame spatial density. Visual tokens are scored relative to their frame-level semantic center.
3. Global token sampling. Temporal and spatial signals form a unified token density, and density-guided FPS selects tokens under a global budget.

Unless otherwise specified, the paper uses sigma_t=1.0 and alpha=0.5 for all models, benchmarks, and retention ratios.

🤖 Supported Models

Backbone	Selection space	Visual tokens passed to the LLM
LLaVA-OneVision-7B	Projected and spatially pooled video features	Selected pooled tokens with the native video newline token
Qwen2.5-VL-7B	FP32 mean-pooled pre-merger features, restored from window order	Corresponding native post-merger embeddings
Qwen3-VL-8B	FP32 mean-pooled pre-merger features	Corresponding native post-merger embeddings with synchronized DeepStack indices

Pruning is applied only to video inputs. Image inputs are unchanged. The Qwen integrations prune during single-sample prefill and preserve the model's native serving embeddings, positional information, and aligned auxiliary streams.

🛠️ Installation

Clone the repository:

git clone https://github.com/yeluoy/GSTEP.git
cd GSTEP

The following setup uses Python 3.10, PyTorch 2.5.1, and CUDA 12.1. Select the dependency file for the model family you intend to evaluate.

LLaVA-OneVision

conda create -n gstep python=3.10 -y
conda activate gstep
python -m pip install --upgrade pip
python -m pip install torch==2.5.1 torchvision==0.20.1 \
  --index-url https://download.pytorch.org/whl/cu121
python -m pip install -r requirements.txt

Qwen2.5-VL and Qwen3-VL

conda create -n gstep python=3.10 -y
conda activate gstep
python -m pip install --upgrade pip
python -m pip install torch==2.5.1 torchvision==0.20.1 \
  --index-url https://download.pytorch.org/whl/cu121
python -m pip install -r requirements-qwen.txt

FlashAttention-2 is recommended for efficient evaluation:

python -m pip install flash-attn==2.8.3 --no-build-isolation

If FlashAttention-2 is unavailable, set ATTN_IMPLEMENTATION=sdpa when running the evaluation scripts. The optional CUDA FPS backend additionally requires a CUDA toolkit with nvcc; the default PyTorch backend does not.

Dataset locations and Hugging Face settings can be supplied through shell environment variables or a repository-level .env file. See .env.example for the available entries, and never commit access tokens.

📊 Evaluation

R_RATIO denotes the visual-token retention ratio:

• R_RATIO=0.25 retains 25% of visual tokens, corresponding to 75% pruning.
• R_RATIO=0.15 retains 15% of visual tokens, corresponding to 85% pruning.

The paper configuration is used by default:

Variable	Default	Description
R_RATIO	0.25	Visual-token retention ratio
GSTEP_SIGMA_T	1.0	Temporal Gaussian smoothing scale
GSTEP_ALPHA	0.5	Coverage-density trade-off
GSTEP_FPS_BACKEND	torch	FPS implementation used for accuracy evaluation

LLaVA-OneVision-7B

Run one benchmark:

CUDA_VISIBLE_DEVICES=0 NUM_PROCESSES=1 \
  bash scripts/run_gstep_single.sh longvideobench_val_v 0.25

Run the four benchmark and four retention-ratio sweep:

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5 NUM_PROCESSES=6 \
  TASKS=videomme,mlvu_dev,longvideobench_val_v,mvbench \
  R_RATIO_LIST=0.25,0.20,0.15,0.10 \
  bash scripts/run_gstep_sweep.sh

Qwen2.5-VL-7B

CUDA_VISIBLE_DEVICES=0 NUM_PROCESSES=1 \
  bash scripts/run_gstep_qwen25_single.sh videomme 0.25

CUDA_VISIBLE_DEVICES=0,1,2,3 NUM_PROCESSES=4 \
  TASKS=videomme,mlvu_dev,longvideobench_val_v,mvbench \
  R_RATIO_LIST=0.25,0.20,0.15,0.10 \
  bash scripts/run_gstep_qwen25_sweep.sh

Qwen3-VL-8B

CUDA_VISIBLE_DEVICES=0 NUM_PROCESSES=1 \
  bash scripts/run_gstep_qwen3_single.sh videomme 0.25

CUDA_VISIBLE_DEVICES=0,1 NUM_PROCESSES=2 \
  TASKS=videomme,mlvu_dev,longvideobench_val_v,mvbench,egoschema \
  R_RATIO_LIST=0.25,0.20,0.15,0.10 \
  bash scripts/run_gstep_qwen3_sweep.sh

Set LIMIT=1 for a short connectivity check. Evaluation results and console logs are written under logs/.

⚙️ FPS Backends

The default torch backend implements density-guided FPS with standard PyTorch operations and is used for accuracy reproduction.

Set GSTEP_FPS_BACKEND=auto to use the CUDA extension when it is available and the similarity matrix fits in memory, with automatic fallback to PyTorch. Set GSTEP_FPS_BACKEND=cuda to require the CUDA path. The extension is compiled on first use and cached through PyTorch's standard extension mechanism.

Both backends implement the same selection rule. Near-tied scores can produce different retained indices because of floating-point execution order, so use the same backend when comparing exact benchmark results.

⏱️ Efficiency

The paper reports the following results on LLaVA-OneVision-7B and LongVideoBench using six NVIDIA A100 40GB GPUs and FlashAttention-2:

Retention	Pruning	E2E (s)	LLM time (s)	GPU (GB)	KV cache (MB)
1.00	0%	907.5	280.3	17.57	340
0.25	75%	777.5	173.6	16.00	90
0.15	85%	754.8	157.7	15.80	55

Run the corresponding GSTEP configurations with:

bash scripts/run_gstep_efficiency.sh

The script defaults to TORCH_CUDA_ARCH_LIST=8.0 for A100 GPUs. Set the appropriate architecture value when using a different GPU.

Acknowledgments

This codebase builds on VidCom2, LLaVA-NeXT, and LMMs-Eval.

📚 Citation

If you find GSTEP useful in your research, please consider citing our paper:

@article{zhang2026gstep,
  title={GSTEP: Global Spatio-Temporal Density-Driven Visual Token Pruning for Efficient Video Large Language Models},
  author={Zhang, Mengjie and Zhu, Qihui and Zhang, Tao and Chen, Shuangwu and Qin, Huihuang and Guo, Yu and Ye, Shenghao and Wen, Zijian and Hou, Yunpeng and Jin, Dong and others},
  journal={arXiv preprint arXiv:2608.03083},
  year={2026}
}