Awesome-Efficient-Inference-for-LRMs

June 13, 2026 · View on GitHub

Awesome-Efficient-Inference-for-LRMs is a collection of state-of-the-art, novel, exciting, token-efficient methods for Large Reasoning Models (LRMs). It contains papers, codes, datasets, evaluations, and analyses. Any additional things regarding efficient inference for LRMs, PRs, and issues are welcome, and we are glad to add you to the contributor list here. Any problems, please contact yliu@u.nus.edu. Our survey paper is online: Efficient Inference for Large Reasoning Models: A Survey. If you find this repository useful to your research or work, it is really appreciated if to star this repository and cite our papers here. :sparkles:

overview

Update

  • (2026/06/07) Added 40+ new papers (2025.05–2026.06) across Explicit Compact CoT, Implicit Latent CoT, Limitations & Challenges, Further Improvement, and Survey.
  • (2026/05/31) Our survey paper has been accepted by the IEEE T-PAMI 2026.
  • (2025/03/29) Our survey paper is online: Efficient Inference for Large Reasoning Models: A Survey.

Reference

If you find this repository helpful for your research, we would greatly appreciate it if you could cite our papers. :sparkles:

@article{liu2025efficient,
  title={Efficient Inference for Large Reasoning Models: A Survey},
  author={Liu, Yue and Wu, Jiaying and He, Yufei and Gao, Hongcheng and Chen, Hongyu and Bi, Baolong and Zhang, Jiaheng and Huang, Zhiqi and Hooi, Bryan},
  journal={arXiv preprint arXiv:2503.23077},
  year={2025}
}

Bookmarks

Papers

Survey

TimeTitleVenuePaperCode
2026.05Efficient Inference for Large Reasoning Models: A SurveyIEEE T-PAMI'26linklink
2025.08Don't Overthink It: A Survey of Efficient R1-style Large Reasoning ModelsarXiv'25link-
2025.07Towards Concise and Adaptive Thinking in Large Reasoning Models: A SurveyarXiv'25link-
2025.07Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMsarXiv'25link-
2025.04Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought ReasoningarXiv'25linklink
2025.04Efficient Reasoning Models: A SurveyarXiv'25linklink
2025.03Efficient Inference for Large Reasoning Models: A SurveyarXiv'25linklink
2025.03Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language ModelsarXiv'25linklink
2025.03A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and BeyondarXiv'25linklink
2025.03Stop Overthinking: A Survey on Efficient Reasoning for Large Language ModelsarXiv'25linklink

Explicit Compact CoT

TimeTitleVenuePaperCode
2026.05Stop When Reasoning Converges: Semantic-Preserving Early Exit for Reasoning ModelsarXiv'26linklink
2026.03Efficient Reasoning with Balanced Thinking (ReBalance)arXiv'26link-
2026.03Draft-Thinking: Learning Efficient Reasoning in Long Chain-of-Thought LLMsarXiv'26link-
2026.02Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought CompressionarXiv'26link-
2026.02Self-Verification Dilemma: Experience-Driven Suppression of Overused Checking in LLM ReasoningarXiv'26link-
2026.02Short Chains, Deep Thoughts: Balancing Reasoning Efficiency and Intra-Segment Capability via Split-Merge OptimizationarXiv'26link-
2026.01Mitigating Overthinking in Large Reasoning Models via Difficulty-aware Reinforcement LearningarXiv'26link-
2025.10DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement LearningarXiv'25link-
2025.10Learning to Reason Efficiently with Discounted Reinforcement LearningarXiv'25link-
2025.10Beyond Token Length: Step Pruner for Efficient and Accurate Reasoning in Large Language ModelsarXiv'25link-
2025.09Your Models Have Thought Enough: Training Large Reasoning Models to Stop OverthinkingarXiv'25link-
2025.08Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step EntropyarXiv'25linklink
2025.08BudgetThinker: Empowering Budget-aware LLM Reasoning with Control TokensarXiv'25link-
2025.08Efficient Reasoning for Large Reasoning Language Models via Certainty-Guided Reflection SuppressionarXiv'25link-
2025.07Think Clearly: Improving Reasoning via Redundant Token PruningarXiv'25link-
2025.06AALC: Large Language Model Efficient Reasoning via Adaptive Accuracy-Length ControlarXiv'25link-
2025.06Steering LLM Thinking with Budget GuidancearXiv'25linklink
2025.06Exploring and Exploiting the Inherent Efficiency within Large Reasoning Models for Self-Guided Efficiency EnhancementarXiv'25link-
2025.05REA-RL: Reflection-Aware Online Reinforcement Learning for Efficient ReasoningarXiv'25linklink
2025.05Can Pruning Improve Reasoning? Revisiting Long-CoT Compression with Capability in Mind for Better ReasoningarXiv'25link-
2025.05Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language ModelsarXiv'25linklink
2025.05ConCISE: Confidence-guided Compression in Step-by-step Efficient ReasoningarXiv'25link-
2025.05Scalable Chain of Thoughts via Elastic ReasoningarXiv'25link-
2025.05S-GRPO: Early Exit via Reinforcement Learning in Reasoning ModelsarXiv'25link-
2025.05Making Small Language Models Efficient Reasoners: Intervention, Supervision, ReinforcementarXiv'25link-
2025.05Accelerating Chain-of-Thought Reasoning: When Goal-Gradient Importance Meets Dynamic SkippingarXiv'25link-
2025.05SelfBudgeter: Adaptive Token Allocation for Efficient LLM ReasoningarXiv'25link-
2025.05Not All Thoughts are Generated Equal: Efficient LLM Reasoning via Multi-Turn Reinforcement LearningarXiv'25linklink
2025.05Fractured Chain-of-Thought ReasoningarXiv'25link-
2025.05Efficient RL Training for Reasoning Models via Length-Aware OptimizationarXiv'25link-
2025.05DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning ModelsarXiv'25link-
2025.05FlashThink: An Early Exit Method For Efficient ReasoningarXiv'25link-
2025.05Optimizing Anytime Reasoning via Budget Relative Policy OptimizationarXiv'25linklink
2025.05VeriThinker: Learning to Verify Makes Reasoning Model EfficientarXiv'25linklink
2025.05Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM ReasoningarXiv'25linklink
2025.05ThinkLess: A Training-Free Inference-Efficient Method for Reducing Reasoning RedundancyarXiv'25link-
2025.05Learn to Reason Efficiently with Adaptive Length-based Reward ShapingarXiv'25linklink
2025.05R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and SearcharXiv'25linklink
2025.05Incentivizing Dual Process Thinking for Efficient Large Language Model ReasoningarXiv'25link-
2025.05ConciseRL: Conciseness-Guided Reinforcement Learning for Efficient Reasoning ModelsarXiv'25linklink
2025.05TrimR: Verifier-based Training-Free Thinking Compression for Efficient Test-Time ScalingarXiv'25link-
2025.05Not All Tokens Are What You Need In ThinkingarXiv'25linklink
2025.05LIMOPro: Reasoning Refinement for Efficient and Effective Test-time ScalingarXiv'25linklink
2025.05Walk Before You Run! Concise LLM Reasoning via Reinforcement LearningarXiv'25linklink
2025.05CoThink: Token-Efficient Reasoning via Instruct Models Guiding Reasoning ModelsarXiv'25link-
2025.05Don't Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning ModelsarXiv'25link-
2025.05A*-Thought: Efficient Reasoning via Bidirectional Compression for Low-Resource SettingsarXiv'25linklink
2025.05Efficient Reasoning via Chain of Unconscious ThoughtarXiv'25linklink
2025.04Syzygy of Thoughts: Improving LLM CoT with the Minimal Free ResolutionarXiv'25linklink
2025.03Sketch-of-thought: Efficient llm reasoning with adaptive cognitive-inspired sketching (SoT)arXiv'25linklink
2025.03SOLAR: Scalable Optimization of Large-scale Architecture for ReasoningarXiv'25link-
2025.03InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language ModelsarXiv'25link-
2025.03L1: Controlling How Long A Reasoning Model Thinks With Reinforcement LearningarXiv'25linklink
2025.03Optimizing Test-Time Compute via Meta Reinforcement Fine-TuningarXiv'25linklink
2025.02Chain of Draft: Thinking Faster by Writing LessarXiv'25linklink
2025.02Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language ModelsarXiv'25link-
2025.02TokenSkip: Controllable Chain-of-Thought Compression in LLMsarXiv'25linklink
2025.02LightThinker: Thinking Step-by-Step CompressionarXiv'25linklink
2025.02CoT-Valve: Length-Compressible Chain-of-Thought TuningarXiv'25link-
2025.02Self-Training Elicits Concise Reasoning in Large Language ModelsarXiv'25linklink
2025.02DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft TokensarXiv'25link-
2025.02Training Language Models to Reason EfficientlyarXiv'25linklink
2025.02Anthropic. Claude 3.7 sonnet and claude codeAnthropic'25link-
2025.02Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language ModelsarXiv'25link-
2025.01Kimi k1.5: Scaling Reinforcement Learning with LLMsarXiv'25link-
2025.01O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning PruningarXiv'25linklink
2025.01Think Smarter not Harder: Adaptive Reasoning with Inference Aware OptimizationarXiv'25link-
2024.12C3oT: Generating Shorter Chain-of-Thought without Compromising EffectivenessarXiv'24link-
2024.12Token-Budget-Aware LLM ReasoningarXiv'24linklink
2024.12Verbosity-Aware Rationale Reduction: Effective Reduction of Redundant Rationale via Principled CriteriaarXiv'24link-
2024.11Can Language Models Learn to Skip Steps?arXiv'24linklink
2024.07Concise Thoughts: Impact of Output Length on LLM Reasoning and CostarXiv'24link-
2024.07Distilling System 2 into System 1arXiv'24link-
takeaway1

Implicit Latent CoT

TimeTitleVenuePaperCode
2026.05Selective Latent Thinking: Adaptive Compression of LLM Reasoning ChainsarXiv'26linklink
2026.05LatentRAG: Latent Reasoning and Retrieval for Efficient Agentic RAGarXiv'26link-
2026.04Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-ThoughtarXiv'26link-
2026.02LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut ModulationarXiv'26link-
2025.10KaVa: Latent Reasoning via Compressed KV-Cache DistillationarXiv'25link-
2025.05Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept SpacearXiv'25linklink
2025.05Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning ChainsarXiv'25linklink
2025.02CODI: Compressing Chain-of-Thought into Continuous Space via Self-DistillationarXiv'25link-
2025.02Token Assorted: Mixing Latent and Text Tokens for Improved Language Model ReasoningarXiv'25link-
2025.02SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMsarXiv'25link-
2025.01Efficient Reasoning with Hidden ThinkingarXiv'25linklink
2024.12Training Large Language Models to Reason in a Continuous Latent SpacearXiv'24link-
2024.12Compressed Chain of Thought: Efficient Reasoning Through Dense RepresentationsarXiv'24link-
2024.05From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by SteparXiv'24linklink
2023.11Implicit Chain of Thought Reasoning via Knowledge DistillationarXiv'23linklink
takeaway2

Limitations and Challenges

User-centric Controllable Reasoning

TimeTitleVenuePaperCode
2026.02Conformal Thinking: Risk Control for Reasoning on a Compute BudgetarXiv'26link-
2025.10e1: Learning Adaptive Control of Reasoning EffortarXiv'25link-
2025.05When to Continue Thinking: Adaptive Thinking Mode Switching for Efficient Reasoning (ASRR)arXiv'25link-
2025.02OpenAI o3-mini System CardOpenAI'25link-
2025.02Anthropic. Claude 3.7 sonnet and claude codeAnthropic'25link-

Interpretability of Reasoning

TimeTitleVenuePaperCode
2026.06Unlocking the Black Box of Latent Reasoning: An Interpretability-Guided Approach to InterventionarXiv'26link-
2026.04Are Latent Reasoning Models Easily Interpretable?arXiv'26link-
2026.04LLM Reasoning Is Latent, Not the Chain of ThoughtarXiv'26link-
2024.05FiDeLiS: Faithful Reasoning in Large Language Model for Knowledge Graph Question AnsweringarXiv'24link-
2024.02Challenges and barriers of using large language models (LLM) such as ChatGPT for diagnostic medicine with a focus on digital pathology - a recent scoping reviewDiagnostic pathology'24link-
2024.02(A)I Am Not a Lawyer, But...: Engaging Legal Experts towards Responsible LLM Policies for Legal AdvicearXiv'24link-
2023.12Retrieval-Augmented Generation for Large Language Models: A SurveyarXiv'23linklink
2023.10Contribution and performance of ChatGPT and other Large Language Models (LLM) for scientific and research advancements: a double-edged swordInternational Research Journal of Modernization in Engineering Technology and Science'23link-
2023.08Reasoning in Large Language Models Through Symbolic Math Word ProblemsarXiv'23link-
2021.12Chapter 1. Neural-Symbolic Learning and Reasoning: A Survey and Interpretation1Neuro-Symbolic Artificial Intelligence'21link-

Reasoning Safety

TimeTitleVenuePaperCode
2025.05The First Impression Problem: Internal Bias Triggers Overthinking in Reasoning ModelsarXiv'25link-
2025.03Optimizing Test-Time Compute via Meta Reinforcement Fine-TuningarXiv'25linklink
2025.03Detecting misbehavior in frontier reasoning modelsOpenAI'25link-
2025.02Evaluating the Paperclip Maximizer: Are RL-Based Language Models More Likely to Pursue Instrumental Goals?arXiv'25linklink
2025.01O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning PruningarXiv'25linklink
2025.01GuardReasoner: Towards Reasoning-based LLM SafeguardsarXiv'25linklink
2025.01Kimi k1.5: Scaling Reinforcement Learning with LLMsarXiv'25link-
2024.12C3oT: Generating Shorter Chain-of-Thought without Compromising EffectivenessarXiv'24link-
2024.10FlipAttack: Jailbreak LLMs via FlippingarXiv'24linklink
2023.10Privacy in Large Language Models: Attacks, Defenses and Future DirectionsarXiv'23link-

Broader Application

TimeTitleVenuePaperCode
2025.03SOLAR: Scalable Optimization of Large-scale Architecture for ReasoningarXiv'25link-
2025.03Large language models (LLM) in computational social science: prospects, current state, and challengesSocial Network Analysis and Mining'25link-
2025.03Gemini robotics brings ai into the physical worldGoogle'25link-
2025.03Nvidia isaac gr00t n1: An open foundation model for humanoid robots.Nvidia'25link-
2025.02Anthropic. Claude 3.7 sonnet and claude codeAnthropic'25link-
2025.02TokenSkip: Controllable Chain-of-Thought Compression in LLMsarXiv'25linklink
2025.02RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to ConcretearXiv'25link-
2025.02From Personas to Talks: Revisiting the Impact of Personas on LLM-Synthesized Emotional Support ConversationsarXiv'25link-
2025.02Introducing deep researchOpenAI'25link-
2025.02Introducing gpt-4.5OpenAI'25link-
2025.01Kimi k1.5: Scaling Reinforcement Learning with LLMsarXiv'25link-
2024.12Compressed Chain of Thought: Efficient Reasoning Through Dense RepresentationsarXiv'24link-
2024.08Large Language Model Agent in Financial Trading: A SurveyarXiv'24link-
2024.04Automated Social Science: Language Models as Scientist and SubjectsarXiv'24linklink
2023.11LLM4Drive: A Survey of Large Language Models for Autonomous DrivingarXiv'24linklink
takeaway3

Further Improvement

New Architecture

TimeTitleVenuePaperCode
2026.06Genesis 2: Cascade MoE for Ultra-Efficient CPU InferenceGitHub'26-link
2025.10ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning ModelsICLR'26link-
2025.05R-KV: Redundancy-aware KV Cache Compression for Reasoning ModelsarXiv'25link-
2025.02Large Language Diffusion ModelsarXiv'25linklink
2025.02UniGraph2: Learning a Unified Embedding Space to Bind Multimodal GraphsarXiv'25linklink
2024.03Graph of Thoughts: Solving Elaborate Problems with Large Language ModelsAAAI Conference on Artificial Intelligence'24linklink
2024.02UniGraph: Learning a Unified Cross-Domain Foundation Model for Text-Attributed GraphsarXiv'24linklink
2023.12Mamba: Linear-Time Sequence Modeling with Selective State SpacesarXiv'23linklink
2023.10Language Agent Tree Search Unifies Reasoning Acting and Planning in Language ModelsarXiv'23linklink
2023.05RWKV: Reinventing RNNs for the Transformer EraarXiv'23linklink
2017.10Mastering the game of Go without human knowledgenature'17link-

Model Merge

TimeTitleVenuePaperCode
2026.04Multi-objective Evolutionary Merging Enables Efficient Reasoning ModelsarXiv'26link-
2025.09The Thinking Spectrum: An Empirical Study of Tunable Reasoning in LLMs through Model MergingarXiv'25link-
2025.06Accelerated Test-Time Scaling with Model-Free Speculative SamplingarXiv'25link-
2025.03Unlocking Efficient Long-to-Short LLM Reasoning with Model MergingarXiv'25linklink
2025.03SOLAR: Scalable Optimization of Large-scale Architecture for ReasoningarXiv'25link-
2025.03Optimizing Test-Time Compute via Meta Reinforcement Fine-TuningarXiv'25linklink
2025.02Meta-Reasoner: Dynamic Guidance for Optimized Inference-time Reasoning in Large Language ModelsarXiv'25link-
2025.01Kimi k1.5: Scaling Reinforcement Learning with LLMsarXiv'25link-
2025.01DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement LearningarXiv'25link-
2024.12Token-Budget-Aware LLM ReasoningarXiv'24linklink
2024.08Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and OpportunitiesarXiv'24linklink
2024.07The Llama 3 Herd of ModelsarXiv'24linklink

Agent Router

TimeTitleVenuePaperCode
2026.05Breaking the Reward Barrier: Accelerating Tree-of-Thought Reasoning via Speculative Exploration (SPEX)arXiv'26link-
2026.04Step-GRPO: Internalizing Dynamic Early Exit for Efficient ReasoningarXiv'26link-
2026.04Early Stopping for Large Reasoning Models via Confidence DynamicsarXiv'26link-
2026.03Ares: Adaptive Reasoning Effort Selection for Efficient LLM AgentsarXiv'26link-
2025.09FastTTS: Accelerating Test-Time Scaling for Edge LLM ReasoningASPLOS'26link-
2025.06SPECS: Faster Test-Time Scaling through Speculative DraftsarXiv'25link-
2025.05R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token RoutingarXiv'25linklink
2025.04SpecReason: Fast and Accurate Inference-Time Compute via Speculative ReasoningarXiv'25linklink
2025.02Confident or Seek Stronger: Exploring Uncertainty-Based On-device LLM Routing From Benchmarking to GeneralizationarXiv'25link-
2025.01RouteLLM: Learning to Route LLMs from Preference DataThe Thirteenth International Conference on Learning Representations'25link-
2024.10Learning to Route LLMs with Confidence TokensarXiv'24link-
2024.09RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language ModelsThe Thirty-Ninth Annual Conference on Neural Information Processing Systemslink
takeaway4

Contributors

yueliu1999 jiayingwu19 yf-he bhooi Rohan-GRH

(back to top)