Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs

Official PyTorch implementation of the ACMMM 2026 paper: "Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs".

AGS is a training-free, adaptive inference framework designed to balance effectiveness and efficiency in multimodal reasoning. By leveraging a novel vision-to-text attention ratio, AGS dynamically switches between latent perception and explicit reasoning, preserving high-fidelity visual information while maintaining logically structured deduction.

Illustration of AGS. The overall inference process (left) shows the model dynamically shifting its reasoning mode based on the calculated vision-to-text attention ratio.

🚀 News

• [2026-07] 🎉 AGS has been accepted by ACMMM 2026!
• [2026-07] 🔥 Full code and files are released.

💡 Key Highlights

• Attention-Guided Cognitive Decoupling: Introduces a novel vision-to-text attention ratio to distinguish perception-dominant tokens from reasoning-dominant tokens during multimodal inference.
• Adaptive Explicit–Latent Switching: Dynamically routes perceptual steps to the continuous latent space while preserving explicit text generation for logically structured deduction.
• Training-Free & Model-Agnostic: A plug-and-play inference strategy that improves both reasoning accuracy and efficiency across Qwen3-VL-Thinking and InternVL3.5 models from 2B to 8B parameters, without any fine-tuning.

---

🛠️ Installation

1. Clone the repository:

git clone https://github.com/swordAndSnow/MM26-AGS.git
cd MM26-AGS

2. Create environment and install dependencies:

conda create -n AGS python=3.10 -y
conda activate AGS
pip install --upgrade pip
pip install -r requirements.txt

📦 Data & Model Preparation and Evaluation

please check the run_bash.sh.

📈 Main Results

📝 Acknowledgments

Our implementation is built upon the foundational architectures of SwiReasoning. We sincerely thank the authors for open-sourcing their incredible works.

Contact

If you have any question, you can raise an issue or email Haoqian Kang (13703206989@163.com or 26B951036@stu.hit.edu.cn).