XMCC: Explainable Multimodal CoT Compressor

The repository of "Bridging Efficiency and Transparency: Explainable CoT Compression in Multimodal Large Reasoning Models"

💡 Overview

We introduce XMCC (eXplainable Multimodal CoT Compressor), a novel framework that bridges efficiency and transparency in multimodal reasoning by compressing verbose chains of thought while preserving critical reasoning steps and core visual-language alignment cues. XMCC formulates compression as a sequential decision-making process optimized via reinforcement learning, and simultaneously generates natural-language explanations for its compression decisions.

⚡ Highlights

1. Three-Stage Pipeline

   • Stage I: Long CoT Synthesis
     Generate diverse reasoning trajectories using M=3 heterogeneous MLRMs (Qwen3-VL, InternVL3.5, Kimi-VL) with K=2 stochastic samples per model → N=6 CoTs per instance.

   • Stage II: Explainable Compressor Training
     Train a lightweight compressor (Qwen3-VL-2B) via GRPO with a multi-component reward function:

     • $R_\text{fmt}$: Enforces structured output format (<explanation>, <refinement>, and <answer>)
     • $R_\text{acc}$: Preserves answer correctness
     • $R_\text{step}$: Fine-grained evaluation of each reasoning step's contribution to task performance
     • $R_\text{len}$: Adaptive length reward based on task complexity

   • Stage III: SFT for Efficient Reasoning
     Fine-tune base models on compressed CoTs for direct efficient inference.

2. Explainable Compression Decisions: XMCC generates natural-language explanations before producing compressed CoTs, making the compression process transparent and interpretable.

3. Visual Grounding Preservation: Unlike text-based methods that break visual-textual integrity, XMCC explicitly preserves critical visual information and cross-modal alignment cues essential for multimodal reasoning.

📊 Performance

🔍 Differences between text-based CoT compression methods and XMCC

To examine the differences between existing text-based CoT compression methods and XMCC, we analyzed outputs from different compression approaches. Figure (a) shows the compressed CoT produced by a text-based compression method, while (b) shows the result from XMCC. In (a), each "[SKIP]" represents a deleted step. It can be observed that the text-based method erroneously removes critical visually grounded information that defines variable meanings (e.g., "Height of bamboo pole = h_1"''"). In contrast, XMCC preserves these critical alignment cues.

🔍 Case Study on SFT Models

To further investigate the impact of XMCC-compressed data on downstream reasoning performance, we conducted a case study comparing models fine-tuned on different data. Text in the box at the lower left corner is generated by the model fine-tuned on XMCC data, while text in right box is generated by the model fine-tuned on uncompressed CoTs. As shown, models fine-tuned with XMCC refined data produce significantly shorter reasoning chains compared to baseline methods. Moreover, redundant step re-verification and unnecessary self-reflection are notably reduced, which demonstrates the effectiveness of XMCC's compression.

🚀 Quick Start

XMCC's RL post-training and supervised fine-tuning rely on veRL and LLaMA-Factory, respectively. Begin by installing the required frameworks:

# Install veRL for GRPO
git clone https://github.com/volcengine/verl
cd verl && pip install -e .[vllm]

# Install LLaMA-Factory for SFT
git clone https://github.com/hiyouga/LLaMA-Factory.git
cd LLaMA-Factory && pip install -e .

After installation, RL training could be launched with the following commands:

bash /root/xmcc/serve_vllm_auxiliary.sh Qwen/Qwen3-VL-2B-FP8
bash /root/xmcc/scripts/run_qwen3-vl-2b.sh

Upon completion of RL training, the compressor weights could be converted to HuggingFace format using:

cd verl
python3 -m verl.model_merger merge \
    --backend fsdp \
    --local_dir checkpoints/${trainer.project_name}/${trainer.experiment_name}/global_step_1/actor \
    --target_dir checkpoints/${trainer.project_name}/${trainer.experiment_name}/global_step_1/actor/huggingface

After constructing the refined dataset using the compressor, supervised fine-tuning on the base model could be performed with the following command:

cd LLama-Factory
DISABLE_VERSION_CHECK=1 FORCE_TORCHRUN=1 llamafactory-cli train LLama-Factory/examples/train_full/qwen2.5-vl_fullsft.yaml

Finally, we provide evaluation scripts for Explanation Quality and Visual Information Preservation assessment. The corresponding prompts are also included in the scripts.

python ./eval_vis.py \
    -d <your_data_path>
    -o <output_path>
    -m <served_model_name>
    -p <port>

  python ./eval_expl.py \
    -d <your_data_path>
    -o <output_path>
    -m <served_model_name>
    -p <port>