readme.md

April 17, 2026 · View on GitHub

DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Models

This is the official repository of the paper DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Models.

News: Our paper has been accepted by ACL 2026 (Findings)!

  • If you find our work helpful and it has been of any assistance to you, we would greatly appreciate it if you could kindly cite it:
@article{jiang2025drp,
  title={DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Models},
  author={Jiang, Yuxuan and Li, Dawei and Ferraro, Frank},
  journal={arXiv preprint arXiv:2505.13975},
  year={2025}
}

🚀 Introduction

While Large Reasoning Models (LRMs) have demonstrated remarkable success in complex reasoning tasks through Long Chain-of-Thought (CoT) reasoning, their inference often involves excessively verbose reasoning traces, resulting in substantial inefficiency. To address this issue, we propose \textbf{Distilled Reasoning Pruning (DRP)}, a hybrid framework that combines inference-time pruning with tuning-based distillation---two widely used strategies for efficient reasoning. DRP employs a teacher model to perform skill-aware step decomposition and content pruning, and then distills the pruned reasoning paths into a student model, enabling it to reason both efficiently and accurately. Across a series of challenging mathematical reasoning datasets, we find models trained with DRP achieve substantial improvements in token efficiency without sacrificing accuracy. Specifically, DRP reduces the average token usage on GSM8K from 917 to 328 while improving accuracy from 91.7% to 94.1%, and achieves a 43% token reduction on AIME with no performance drop. Further analysis reveals that aligning the reasoning structure of training CoTs with the student’s reasoning capacity is critical for effective knowledge transfer and performance gains.

📄 Get Started

📝 Setup

  • Firstly, install the required environment:
# Training env (for LoRA)
python -m venv train_env
source train_env/bin/activate
pip install -r requirements/train.txt

# Evaluation env 
python -m venv eval_env
source eval_env/bin/activate
pip install -r requirements/eval.txt

💻 Models

We use R1-Distill-Qwen-7B for our main experiment. Please first get the access of that model.

📥 Data

Coming soon.

⛳️ Run

Reasoning Path Decomposition and Pruning

  • Run the following command:
  python3 split_verify.py

Main Experiment

  • With the training data prepared:
  • First run the following command to train the student models:
  bash scripts/run_all.sh

Acknowledge