MixReasoning: Switching Modes to Think

Think deeply only when it matters. MixReasoning dynamically switches between detailed and concise reasoning within a single response, reducing reasoning tokens by 13%-49% while maintaining task performance.

MixReasoning switches modes by adjusting the concise LoRA strength: low-confidence windows trigger thinking mode; once confidence recovers, it returns to concise mode.

Overview

This repository contains the official research code for MixReasoning (ICML 2026), a model-agnostic framework for difficulty-aware reasoning-mode switching. It includes concise-mode LoRA training, a customized vLLM backend, OpenAI-compatible inference clients, benchmark generation, and offline evaluation.

MixReasoning builds on two ideas:

• How to switch: attach a lightweight concise LoRA adapter to a base reasoning model and change the active mode during decoding. The base model preserves detailed reasoning, while the adapter provides concise reasoning.
• When to switch: estimate local difficulty from sliding-window token confidence. Low-confidence segments receive more detailed reasoning; high-confidence routine segments use concise reasoning. Separate low and high thresholds stabilize the transitions.

Across AIME 2024, MATH-500, GPQA-Diamond, and GSM8K, MixReasoning reduces generated reasoning tokens by 13%-49% while matching or improving the performance of the corresponding base reasoning models.

Installation

We recommend using a new environment with Python 3.10+.

conda create -n mixreason python=3.10 -y
conda activate mixreason

git clone --recurse-submodules https://github.com/haiquanlu/MixReasoning.git
cd MixReasoning

python -m pip install --upgrade pip
python -m pip install -r requirements.txt

MixReasoning is implemented on top of a modified vLLM fork. Choose one installation method below.

Option 1: reuse the matching precompiled CUDA extensions:

VLLM_PRECOMPILED_WHEEL_LOCATION=https://wheels.vllm.ai/f32a5bc5058afc2fb601dcb456b581e2fefa94dd/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl \
  python -m pip install --editable ./vllm

Option 2: build the customized vLLM from source:

python -m pip install --editable ./vllm

If you cloned the repository without --recurse-submodules, initialize the submodule first:

git submodule update --init --recursive

Quick Start

Run all inference and evaluation commands from the repository root.

1. Download the concise adapter

The released Qwen3-8B concise adapter is available from the MixReasoning Hugging Face collection:

hf download haiquanlu/Qwen3-8B-Concise \
  --local-dir lora_weights/Qwen3-8B-Concise

export CONCISE_LORA="$PWD/lora_weights/Qwen3-8B-Concise"

You can instead set CONCISE_LORA to a checkpoint trained with the instructions below.

2. Start the customized vLLM server

export CUDA_VISIBLE_DEVICES=0,1,2,3

vllm serve Qwen/Qwen3-8B \
  --tensor-parallel-size 4 \
  --enable-prefix-caching \
  --port 8000 \
  --gpu-memory-utilization 0.9 \
  --return-tokens-as-token-ids \
  --enable-lora \
  --max-lora-rank 32 \
  --lora-modules "concise=$CONCISE_LORA"

For a different number of GPUs, update both CUDA_VISIBLE_DEVICES and --tensor-parallel-size.

3. Generate one MixReasoning response

Open a second terminal, activate the inference environment, and run:

conda activate mixreason
cd MixReasoning

python -u run_main.py \
  --model Qwen/Qwen3-8B \
  --dataset math500 \
  --index 0 \
  --port 8000 \
  --max-tokens 16384 \
  --enable-mixreaoning \
  --conf_group_size 16 \
  --thres_low 14 \
  --thres_high 16 \
  --save-results \
  --exp mixreasoning_14-16_g16 \
  --repeat_exp_num 0

This enables confidence-aware switching to the concise adapter. The server-side switching window is 16 tokens; --conf_group_size 16 uses the same window for confidence values saved with the result.

Supported dataset names are gsm8k, aime24, aime25, math500, and gpqa_diamond. The first four are downloaded through Hugging Face Datasets; GPQA-Diamond uses data/gpqa-diamond.json.

scripts/run.sh is a high-concurrency sweep template. Adjust its dataset, valid index range, thresholds, and concurrency before launching it; MATH-500 uses indices 0-499.

Concise Mode Training

Use a separate environment for training because train_lora/requirements.txt includes upstream vLLM and would overwrite the customized inference installation.

conda create -n mixreason-train python=3.10 -y
conda activate mixreason-train

cd MixReasoning
python -m pip install --upgrade pip
python -m pip install -r train_lora/requirements.txt
cd train_lora

The command below matches the paper configuration. Run it from train_lora/ so the relative DeepSpeed configuration path resolves correctly.

export WANDB_PROJECT=MixReasoning
# Use this instead if you do not want online W&B logging:
# export WANDB_MODE=disabled

deepspeed --include localhost:0,1,2,3 lora_ft.py \
  --model_name Qwen/Qwen3-8B \
  --dataset mix-DeepScaleR \
  --learning_rate 1e-5 \
  --wd 0.01 \
  --lora_rank 32 \
  --lora_alpha 64 \
  --target all \
  --batch_size 16 \
  --gradient_accumulation_steps 1 \
  --epoch 1 \
  --max_token 8000

The mix-DeepScaleR option loads haiquanlu/mix-DeepscaleR-30k, a short-solution training set derived from DeepScaleR. The code also supports gsm8k.

With the command above, checkpoints are written under:

train_lora/outputs/Qwen3-8B/mix-DeepScaleR/*

Point CONCISE_LORA in the Quick Start to the desired checkpoint directory.

Evaluation

Generate responses

Use scripts/run.sh to generate model responses. The currently supported datasets are gsm8k, aime24, math500, aime25, and gpqa_diamond.

Set DATASET and the corresponding sample index range in scripts/run.sh, adjust the model and switching hyperparameters if needed, and run the launcher from the repository root:

bash scripts/run.sh

Generated responses and token-level switching information are saved as CSV files under results/.

Evaluate generated responses

Evaluate a saved CSV from the repository root:

python -u eval.py \
  --dataset math500 \
  --csv-file results.csv \
  --model Qwen/Qwen3-8B \
  --exp mixreasoning_14-16_g16 \
  --repeat_exp_num 0

The evaluator reports accuracy, average token count, and incorrect problem indices. It also writes:

experiment_logs/eval-exp-*.log

To aggregate result files labeled with repeat exp IDs, first update the variables in scripts/eval.sh to match your generation settings, then run:

bash scripts/eval.sh

Result CSVs are append-only. Remove or rename an existing result file before rerunning the same example.

Repository layout

MixReasoning/
├── assets/
│   └── method.png                 # Method overview
├── data/
│   └── gpqa-diamond.json          # Preprocessed GPQA-Diamond evaluation data
├── docs/                           # GitHub Pages project site
├── scripts/
│   ├── run_server.sh              # Customized vLLM server launcher template
│   ├── run.sh                     # Benchmark generation sweep template
│   └── eval.sh                    # Five-result evaluation launcher
├── train_lora/
│   ├── config/ds_config.json      # DeepSpeed ZeRO-2 configuration
│   ├── lora_ft.py                 # Concise-mode LoRA SFT entry point
│   ├── requirements.txt           # Training dependencies
│   └── train.sh                   # Multi-GPU training launcher template
├── utils/
│   ├── evaluator.py               # Answer extraction
│   ├── grader.py                  # Mathematical answer equivalence
│   ├── logger.py                  # Evaluation logging
│   ├── math_normalize.py          # Mathematical expression normalization
│   └── utils.py                   # Datasets, metrics, and confidence utilities
├── vllm/                          # Customized vLLM fork (Git submodule)
├── clients.py                     # OpenAI-compatible client and prompt templates
├── run_main.py                    # Single-example MixReasoning generation
├── eval.py                        # Accuracy and token-count evaluation
└── requirements.txt               # Inference and evaluation dependencies

Acknowledgements

This repository builds on vLLM, DeepConf, and DynaSor. We sincerely thank their authors for making these projects available.

Citation

If you find MixReasoning useful, please cite:

@inproceedings{lu2026mixreasoning,
  title     = {MixReasoning: Switching Modes to Think},
  author    = {Lu, Haiquan and Fang, Gongfan and Ma, Xinyin and Li, Qi and Wang, Xinchao},
  booktitle = {Proceedings of the 43rd International Conference on Machine Learning},
  year      = {2026}
}