🔰 LLM Survey Papers Collection
April 7, 2025 · View on GitHub
This repo contains a category wise collection of 200+ LLM survey papers.
Quick links
Related Repositories
- 🧱LLM Engineer Toolkit - Contains a curated list of 120+ LLM libraries category wise.
- 🚀RAG Zero to Hero Guide - Comprehensive guide to learn RAG from basics to advanced.
General
| Paper | Link |
|---|---|
| Large Language Models: A Survey | Link |
| A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4 | Link |
| A Comprehensive Overview of Large Language Models | Link |
| A Survey of Large Language Models | Link |
| Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects | Link |
Prompting Techniques
| Paper | Link |
|---|---|
| The Prompt Report: A Systematic Survey of Prompting Techniques | Link |
| Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization | Link |
| Efficient Prompting Methods for Large Language Models: A Survey | Link |
| A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications | Link |
| In-context Learning with Retrieved Demonstrations for Language Models: A Survey | Link |
Small Language Models
| Paper | Link |
|---|---|
| Small Language Models (SLMs) Can Still Pack a Punch: A survey | Link |
| A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthiness | Link |
| A Survey of Small Language Models | Link |
| Small Language Models: Survey, Measurements, and Insights | Link |
Multi Modal LLMs
| Paper | Link |
|---|---|
| Surveying the MLLM Landscape: A Meta-Review of Current Surveys | Link |
| From Specific-MLLM to Omni-MLLM: A Survey about the MLLMs alligned with Multi-Modality | Link |
| How to Bridge the Gap between Modalities: Survey on Multimodal Large Language Model | Link |
| Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey | Link |
| MM-LLMs: Recent Advances in MultiModal Large Language Models | Link |
| A Survey on Multimodal Large Language Models | Link |
Multilingual LLMs
| Paper | Link |
|---|---|
| Multilingual Large Language Models: A Systematic Survey | Link |
| A Survey of Large Language Models for European Languages | Link |
| A Survey on Large Language Models with Multilingualism: Recent Advances and New Frontiers | Link |
| Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers | Link |
| A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias | Link |
Data Management
| Paper | Link |
|---|---|
| A Survey on Data Selection for Language Models | Link |
| Datasets for Large Language Models: A Comprehensive Survey | Link |
| A Survey on Data Selection for LLM Instruction Tuning | Link |
| Data Management For Training Large Language Models: A Survey | Link |
Post Training
| Paper | Link |
|---|---|
| A Survey on Post-training of Large Language Models | Link |
| LLM Post-Training: A Deep Dive into Reasoning Large Language Models | Link |
| Reinforcement Learning Enhanced LLMs: A Survey | Link |
| Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models | Link |
| A Comprehensive Survey of LLM Alignment Techniques: RLHF, RLAIF, PPO, DPO and More | Link |
| Towards Scalable Automated Alignment of LLMs: A Survey | Link |
| Large Language Model Alignment: A Survey | Link |
| Instruction Tuning for Large Language Models: A Survey | Link |
LLM Fine-Tuning
| Paper | Link |
|---|---|
| Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies | Link |
| Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey | Link |
| Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment | Link |
LLM Inference
| Paper | Link |
|---|---|
| A Survey on LLM Inference-Time Self-Improvement | Link |
| A Comprehensive Survey of Accelerated Generation Techniques in Large Language Models | Link |
| Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems | Link |
| A Thorough Examination of Decoding Methods in the Era of LLMs | Link |
| Faster and Lighter LLMs: A Survey on Current Challenges and Way Forward | Link |
LLM Architectures
| Paper | Link |
|---|---|
| A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications | Link |
| A Survey on Mixture of Experts | Link |
LLM Abilities and Limitations
| Paper | Link |
|---|---|
| Emergent Abilities in Large Language Models: A Survey | Link |
| A Survey on Multi-Turn Interaction Capabilities of Large Language Models | Link |
| A Survey on Large Language Models with some Insights on their Capabilities and Limitations | Link |
| Knowledge Boundary of Large Language Models: A Survey | Link |
| A Survey on the Honesty of Large Language Models | Link |
LLM Reasoning
| Paper | Link |
|---|---|
| A Survey on Enhancing Causal Reasoning Ability of Large Language Models | Link |
| Reasoning Language Models: A Blueprint | Link |
| Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models | Link |
| A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges | Link |
| A Survey of Table Reasoning with Large Language Models | Link |
| Large Language Models for Mathematical Reasoning: Progresses and Challenges | Link |
| Towards Reasoning in Large Language Models: A Survey | Link |
Chain of Thought
| Paper | Link |
|---|---|
| Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents | Link |
| Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future | Link |
| Towards Better Chain-of-Thought Prompting Strategies: A Survey | Link |
LLM RAG
| Paper | Link |
|---|---|
| Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG | Link |
| A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models | Link |
| Exploring AI Text Generation, Retrieval-Augmented Generation, and Detection Technologies: a Comprehensive Overview | Link |
| Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wisely | Link |
| Contextual Compression in Retrieval-Augmented Generation for Large Language Models: A Survey | Link |
| Trustworthiness in Retrieval-Augmented Generation Systems: A Survey | Link |
| Graph Retrieval-Augmented Generation: A Survey | Link |
| Evaluation of Retrieval-Augmented Generation: A Survey | Link |
| A Survey on Knowledge-Oriented Retrieval-Augmented Generation | Link |
| Retrieval-Augmented Generation for AI-Generated Content: A Survey | Link |
LLM Agents
| Paper | Link |
|---|---|
| Survey on Evaluation of LLM-based Agents | Link |
| GUI Agents: A Survey | Link |
| Large Language Model-Brained GUI Agents: A Survey | Link |
| GUI Agents with Foundation Models: A Comprehensive Survey | Link |
| The Rise and Potential of Large Language Model Based Agents: A Survey | Link |
| A Survey on Large Language Model-based Agents for Statistics and Data Science | Link |
| Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents | Link |
| Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems | Link |
| A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios | Link |
| From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agents | Link |
| Agents in Software Engineering: Survey, Landscape, and Vision | Link |
| Large Language Model-Based Agents for Software Engineering: A Survey | Link |
| From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Future | Link |
| Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Security | Link |
| Large Language Model Agent in Financial Trading: A Survey | Link |
| Tool Learning with Large Language Models: A Survey | Link |
| What Are Tools Anyway? A Survey from the Language Model Perspective | Link |
| Large Language Model based Multi-Agents: A Survey of Progress and Challenges | Link |
| Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects | Link |
Data Augmentation
| Paper | Link |
|---|---|
| Text Data Augmentation for Large Language Models: A Comprehensive Survey of Methods, Challenges, and Opportunities | Link |
| Data Augmentation using LLMs: Data Perspectives, Learning Paradigms and Challenges | Link |
| A Survey on Data Augmentation in Large Model Era | Link |
Hallucination
| Paper | Link |
|---|---|
| A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions | Link |
| A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models | Link |
| Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey | Link |
Fairness
| Paper | Link |
|---|---|
| Fairness Definitions in Language Models Explained | Link |
| A survey on fairness of large language models in e-commerce: progress, application, and challenge | Link |
| Fairness in Large Language Models: A Taxonomic Survey | Link |
| A Survey on Fairness in Large Language Models | Link |
Embedding Models
| Paper | Link |
|---|---|
| LLMs are Also Effective Embedding Models: An In-depth Overview | Link |
| When Text Embedding Meets Large Language Model: A Comprehensive Survey | Link |
LLM Evaluation
| Paper | Link |
|---|---|
| LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods | Link |
| From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge | Link |
| A Survey on LLM-as-a-Judge | Link |
| Automatic Metrics in Natural Language Generation: A Survey of Current Evaluation Practices | Link |
| A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations | Link |
| The Challenges of Evaluating LLM Applications: An Analysis of Automated, Human, and LLM-Based Approaches | Link |
| A Survey of Useful LLM Evaluation | Link |
| Leveraging Large Language Models for NLG Evaluation: A Survey | Link |
| Evaluating Large Language Models: A Comprehensive Survey | Link |
| Through the Lens of Core Competency: Survey on Evaluation of Large Language Models | Link |
| A Survey on Evaluation of Large Language Models | Link |
Synthetic Data Generation
| Paper | Link |
|---|---|
| Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models | Link |
| On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey | Link |
| Synthetic Data Generation Using Large Language Models: Advances in Text and Code | Link |
| A Survey on Recent Advances in Conversational Data Generation | Link |
LLM Safety
| Paper | Link |
|---|---|
| Recent advancements in LLM Red-Teaming: Techniques, Defenses, and Ethical Considerations | Link |
| AI Safety in Generative AI Large Language Models: A Survey | Link |
| Jailbreak Attacks and Defenses Against Large Language Models: A Survey | Link |
| Against The Achilles’ Heel: A Survey on Red Teaming for Generative Models | Link |
| On Protecting the Data Privacy of Large Language Models (LLMs): A Survey | Link |
| Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models | Link |
| Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey | Link |
| Building Guardrails for Large Language Models | Link |
| Security and Privacy Challenges of Large Language Models: A Survey | Link |
| A Security Risk Taxonomy for Large Language Models | Link |
| A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation | Link |
LLMs for Specific Domains
| Paper | Domain | Link |
|---|---|---|
| Large language models (LLM) in computational social science: prospects, current state, and challenges | Social Media | Link |
| A Survey on Large Language Models for Code Generation | Coding | Link |
| Large Language Models Meet NL2Code: A Survey | Coding | Link |
| Large Language Models for Education: A Survey | Education | Link |
| Political-LLM: Large Language Models in Political Science | Political Science | Link |
| Code LLMs: A Taxonomy-based Survey | Coding | Link |
| Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Survey | Data Science | Link |
| A Survey of Large Language Models in Finance (FinLLMs) | Finance | Link |
| Large Language Models in Finance: A Survey | Finance | Link |
| Revolutionizing Finance with LLMs: An Overview of Applications and Insights | Finance | Link |
| Legal Evalutions and Challenges of Large Language Models | Legal | Link |
| Large Language Models in Law: A Survey | Legal | Link |
| A Survey for Large Language Models in Biomedicine | Healthcare | Link |
| A Review on Scientific Knowledge Extraction using Large Language Models in Biomedical Sciences | Healthcare | Link |
| Demystifying Large Language Models for Medicine: A Primer | Healthcare | Link |
| Large Language Models for Disease Diagnosis: A Scoping Review | Healthcare | Link |
| A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluations | Healthcare | Link |
| A Survey of Large Language Models in Medicine: Principles, Applications, and Challenges | Healthcare | Link |
| Large Language Models for Medicine: A Survey | Healthcare | Link |
| Large Language Models in Biomedical and Health Informatics: A Bibliometric Review | Healthcare | Link |
| A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directions | Healthcare | Link |
| A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics | Healthcare | Link |
| AI for Biomedicine in the Era of Large Language Models | Healthcare | Link |
| Large Language Models in Bioinformatics: A Survey | Healthcare | Link |
| Large language models for automated scholarly paper review: A survey | Scientific Research | Link |
| LLM4SR: A Survey on Large Language Models for Scientific Research | Scientific Research | Link |
| A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery | Scientific Research | Link |
| Harnessing Large Language Models for Disaster Management: A Survey | Disaster Management | Link |
| Leveraging Foundation Models for Crafting Narrative Visualization: A Survey | Visualization | Link |
| LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights | Security | Link |
| Transformers and Large Language Models for Efficient Intrusion Detection Systems: A Comprehensive Survey | Security | Link |
| A Comprehensive Overview of Large Language Models (LLMs) for Cyber Defences: Opportunities and Directions | Security | Link |
| Mathematical Language Models: A Survey | Maths | Link |
| Software Testing with Large Language Model: Survey, Landscape, and Vision | Software Engineering | Link |
| Large Language Models for Software Engineering: Survey and Open Problems | Software Engineering | Link |
Others
| Paper | Category | Link |
|---|---|---|
| Awes, Laws, and Flaws From Today’s LLM Research | LLM Research | Link |
| Efficient Large Language Models: A Survey | Efficient LLMs | Link |
| Continual Learning for Large Language Models: A Survey | Continual Learning | Link |
| From Selection to Generation: A Survey of LLM-based Active Learning | Active Learning | Link |
| Towards Lifelong Learning of Large Language Models: A Survey | Life Long Learning | Link |
| Privacy-Preserving Large Language Models: Mechanisms, Applications, and Future Directions | Privacy Preserving Mechanism | |
| A Survey on Human Preference Learning for Large Language Models | Human Preference Learning | Link |
| A Survey on Human-Centric LLMs | Human centric LLMs | Link |
| Prompt Compression for Large Language Models: A Survey | Prompts | Link |
| The Oscars of AI Theater: A Survey on Role-Playing with Language Models | Prompts | Link |
| From Natural Language to SQL: Review of LLM-based Text-to-SQL Systems | Text-to-SQL | Link |
| Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL | Text-to-SQL | Link |
| A Survey of NL2SQL with Large Language Models: Where are we, and where are we going? | Text-to-SQL | Link |
| A Survey on Employing Large Language Models for Text-to-SQL Tasks | Text-to-SQL | Link |
| Undesirable Memorization in Large Language Models: A Survey | LLM Memorization | Link |
| A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithms | Low bit LLMs | Link |
| Towards Data Contamination Detection for Modern Large Language Models: Limitations, Inconsistencies, and Oracle Challenges | Data Contamination | Link |
| Benchmark Data Contamination of Large Language Models: A Survey | Data Contamination | Link |
| Attention Heads of Large Language Models: A Survey | LLM Attention Heads | Link |
| Claim Verification in the Age of Large Language Models: A Survey | Fact Checking | Link |
| Generative Large Language Models in Automated Fact-Checking: A Survey | Fact Checking | Link |
| Controllable Text Generation for Large Language Models: A Survey | Controllable Text Generation | Link |
| A Survey on Symbolic Knowledge Distillation of Large Language Models | Knowledge Distillation | Link |
| Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application | Knowledge Distillation | Link |
| Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities | Model Merging | Link |
| Preserving Privacy in Large Language Models: A Survey on Current Threats and Solutions | Privacy Preserving | Link |
| Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption | LLM’s KV-Cache | Link |
| Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models | Collaborative Strategies | Link |
| When Search Engine Services meet Large Language Models: Visions and Challenges | Search Engines meet LLMs | Link |
| A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models | Text Summarization | Link |
| LLMs Meet Multimodal Generation and Editing: A Survey | Multi-modal Generation | Link |
| Large Language Models on Tabular Data - A Survey | Tabular Data | Link |
| Large Language Model for Table Processing: A Survey | Tabular Data | Link |
| Large Language Models for Time Series: A Survey | Time Series Data | Link |
| A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems | Dialogue Systems | Link |
| Advancing Graph Representation Learning with Large Language Models: A Comprehensive Survey of Techniques | Graph Representation Learning | Link |
| When Large Language Models Meet Vector Databases: A Survey | Vector Databases | Link |
| Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models | Context Length | Link |
| The What, Why, and How of Context Length Extension Techniques in Large Language Models – A Detailed Survey | Context Length | Link |
| Leveraging Large Models for Crafting Narrative Visualization: A Survey | Visualization | Link |
| Large Language Models for Social Networks : Applications, Challenges and Solutions | Social Networks | Link |
| Large Language Models for Generative Information Extraction: A Survey | Information Extraction | Link |
| A Survey of Text Watermarking in the Era of Large Language Models | Water Marking | Link |
| Large Language Models on Graphs: A Comprehensive Survey | Graphs | Link |
| A Survey of Graph Meets Large Language Model: Progress and Future Directions | Graphs | Link |
| A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions | LLM Text Detection | Link |
| A Survey on Detection of LLMs-Generated Content | LLM Text Detection | Link |
| Detecting ChatGPT: A Survey of the State of Detecting ChatGPT-Generated Text | LLM Text Detection | Link |
| Large Language Models for Generative Recommendation: A Survey and Visionary Discussions | Recommendations | Link |
| Large Language Models for Information Retrieval: A Survey | Information Retrieval | Link |
| A Survey on Model Compression for Large Language Models | Model Compression | Link |
| A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges | Vision LLMs | Link |
| Knowledge Editing for Large Language Models: A Survey | Knowledge Editing | Link |
| A Comprehensive Survey on Integrating Large Language Models with Knowledge-Based Methods | Knowledge Bases | Link |
⭐️ Star History
Please consider giving a star, if you find this repository useful.