Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-up

💌 Contact us: reversalofthought@163.com

RoT improves reasoning accuracy and efficiency while minimizing computational costs, leveraging Preference-Guided Reverse Reasoning and a Cognitive Preference Manager to optimally explore LLM reasoning with cognitive preferences.

🔥 News

• 2025.05 🎉🎉🎉 Thrilled to share that our Reversal of Thought has been accepted to ACL2025 Main!

  Reversal of Thought enhances LLM reasoning via Preference-Guided Reverse Reasoning and a Cognitive Preference Manager, improving efficiency and interpretability with minimal computational overhead.

• 2025.05 🎉🎉🎉 Thrilled to share that our RoT-based work Less is More [Code] has been accepted to the LLMSR@XLLM ACL 2025 workshop!

  🏆 Achieved 3rd place in the LLMSR@XLLM 2025 Shared Task-III: LLM for Structural Reasoning!

• 2025-04 🎉🎉🎉 RoT demo (v2) demo released with enhanced reasoning stability.

  ⚠️ Note: The instantiation_prompt module was introduced to improve structured reasoning and facilitate clearer extraction of thought processes and answers — this enhancement is not included in the original RoT paper and is unique to the v2 release, featured in our follow-up work Less is More.
  ✅ For better stability and clearer reasoning, we encourage using the v2 demo in both academic and practical settings.

• 2024-12 🎉🎉🎉 Initial RoT demo (v1) released on GitHub!

• 2024-10 🎉🎉🎉 Our paper Reversal of Thought was released on arXiv!

---

📖 Citation

If you find our work useful for your research, please kindly cite our paper as follows:

@inproceedings{yuan2025reversal,
    title = "Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-up",
    author={Yuan, Jiahao and Du, Dehui and Zhang, Hao and Di, Zixiang and Naseem, Usman},
    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    pages = "19442--19459",
    year = "2025"
}

@inproceedings{yuan2025llmsr,
    title = "LLMSR@XLLM25: Less is More: Enhancing Structured Multi-Agent Reasoning via Quality-Guided Distillation",
    author={Yuan, Jiahao and Sun, Xingzhe and Yu, Xing and Wang, Jingwen and Du, Dehui and Cui, Zhiqing and Di, Zixiang},
    booktitle = "Proceedings of the 1st Joint Workshop on Large Language Models and Structure Modeling (XLLM 2025)",
    pages = "274--282",
    year = "2025"
}

---

📚 Dataset

• 24-Game( 1 & 2 ) – 1,000 samples used for evaluation;
• BIG-Bench;
• PythonProgrammingPuzzles;
• MGSM;
• SonnetWriting( 1 & 2 )

---

🚀 Demo for Preference-Guided Reverse Reasoning

🎉🎉🎉 Run reversal_demo.py to see the framework in action:

from utils.llm_utils import *
from utils.prompt import *
pipeline=Pipeline(model_id=model_id, base_url=base_url, api_key=api_key, prob=True)
demos = "Input:... Output:..." #Suggest 2-shot Demos
llm_taste=rot_pipeline( 
  pipeline, reversal_of_thought, demos=demos, warmup=5
)

---

💡 What might reversal_demo.py be used for?

• Enhance LLM-Preferred Prompts for Task Solutions
  Refines prompts to align with LLM-preferred strategies, optimizing task-solving efficiency.

• Potential for Creating Diverse QA Datasets
  Generates varied question-answer pairs to improve dataset diversity.

---

📬 Contact Us

We'd love to hear from you! Whether you have ideas, potential applications, or simply want to chat about this project, feel free to get in touch: 💌 Email: reversalofthought@163.com

---

🙏 Acknowledgement

We gratefully acknowledge the following works, whose insights and frameworks have meaningfully influenced the development of our approach:

• Buffer-of-Thought: Guided us in designing a structured reasoning framework through thought-augmented planning.
• PairS: Motivated us to explore whether LLMs can reason over decisions by leveraging their own learned preference patterns.

@article{yang2024buffer,
  title={Buffer of Thoughts: Thought-Augmented Reasoning with Large Language Models},
  author={Yang, Ling and Yu, Zhaochen and Zhang, Tianjun and Cao, Shiyi and Xu, Minkai and Zhang, Wentao and Gonzalez, Joseph E and Cui, Bin},
  journal={Advances in Neural Information Processing Systems},
  year={2024}
}
@inproceedings{liualigning,
  title={Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators},
  author={Liu, Yinhong and Zhou, Han and Guo, Zhijiang and Shareghi, Ehsan and Vuli{\'c}, Ivan and Korhonen, Anna and Collier, Nigel},
  booktitle={First Conference on Language Modeling},
}

---

📄 License

This project is licensed under the GNU General Public License v3.0 (GPL-3.0).