Knowledge Graphs in RAG.md

July 9, 2026 · View on GitHub

Knowledge Graphs in RAG (Retrieval Augmented Generation)

Note: The definition of KGs may be different in each RAG work. Usually there are 3 types of KGs.
1. Eentity-level KG, while each node represents an entitiy or a concept, and each link represents the semantic relation or self-defined relation between a pair of nodes (e.g., whether they are extracted from the same document). `fine-grained`
2. Metadata-level KG, while each node represents the metadata of a document chunk (e.g., the topic or description of the document chunk), and each link represents self-defined relation between a pair of nodes (e.g., the correlation bewteen the topics). `middle-grained`
3. Document chunk-level KG, while each node represents a document chunk, and each link represents the context sequence of the nodes. `coarse-grained` 
* This is aligned with the of LLM+KG inference:
> Compared with neural networks the captures complex distribution of knowledge, graphs of entities may not be scientifically optimal as a knowledge model.
> But graphs can record the connection among document chunks or the tapologies of entity concepts, to facilitate the *retrieval sequence* of llm inference.

📝 Traditional RAG Frameworks

It is highly recommended that you understand the basic concepts of RAG, and get familiar with one or more of the following frameworks first.

RAG surveys

  1. A Survey on Multimodal Retrieval-Augmented Generation (Arxiv 2025) [Paper] [Discussion (Chinese)] MRAG
  • It summerizes the development of MRAG1.0, MRAG2.0 and MRAG3.0, with discussion about the extraction, parsing, planning and retrival.
  1. Retrieving Multimodal Information for Augmented Generation: A Survey (Arxiv 2023) [Paper] MRAG
  2. Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation (Arxiv 2025) [Paper] [Github] [Discussion (Chinese)] MRAG
  1. AnythingLLM (RAG + AI Agent) [Github] - Default VDB: LanceDB
  2. MaxKB [Github] - Default VDB: PostgreSQL + pgvector
  3. RagFlow (streamlined RAG workflow based on deep document understanding) [Github] - Default VDB: Elasticsearch
  4. Dify (agentic AI workflow, RAG pipeline, agent capabilities, model management, ...) [Github] - VDB: Elasticsearch / TiDB? ([can refer to yaml])
  5. FastGPT (data processing, RAG retrieval, and visual AI workflow orchestration) [Github] - VDB: MongoDB + PostgreSQL (PG Vector)/Milvus
  6. LangChain-Chatchat/Langchain-ChatGLM (local knowledge based LLM RAG and Agent) [Github] Default DB:sqlite?
  7. QAnything (a local knowledge base question-answering system designed to support a wide range of file formats and databases, from 网易有道速读) [Github]
  8. Quivr (Opiniated RAG) [Github] - Any Vectorstore: PGVector, Faiss
  9. RAG-GPT [GitHub]
  10. Verba - [GitHub] VDB：Weaviate
  11. FlashRAG：6 pre-processed benchmark RAG datasets and 16 state-of-the-art RAG algorithms [GitHub] Support multimodal RAG
  12. kotaemon: An open-source clean & customizable RAG UI for chatting with your documents. [GitHub]
  13. RAGapp：Agentic RAG, Built using LlamaIndex [GitHub]
  14. TurboRAG: KVCache+ Chunked text [Paper] [GitHub]
  15. Ten: a voice agent framework to create conversational AI. TEN also facilitates the integration of AI with audio-visual tools, databases, monitoring systems, RAG, and more. [GitHub]
  16. AutoRAG: An Open-Source Framework for RAG Evaluation & Optimization with AutoML-Style Automation [Github]
  17. All RAG Techniques [Github] - A series of simple and clear demos!! Strongly recommended.
  18. Sycamore: AI-powered document processing engine for ETL, RAG, LLM-based applications, and analytics on unstructured data. [Github]

Some good discussions/surveys/tutorials/blogs for beginners to know more about RAG and Graph RAG

  1. Retrieval-Augmented Generation for Large Language Models: A Survey (Arxiv, Mar 2024) [Paper] [Notes for OpenRAG Base]
  2. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (NeurIPS 2020) [Paper]
  3. A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models (Arxiv, Jan 2025) [Paper]

My thoughts!:

  • This may be different from multi-modal RAG, where the context is in multiple modalities (e.g., image, text, video, ...) represented as embedding, and the query is based on fuzzy serach in vector db. If we need to query the knowledge in KGs, we need to rely on other query engine, such as SPAEQL on graphs.
  • Graph RAG/KAG may be more valuable in domain-specific area.
  • However, although the evaluation results on research benchmarks of multi-hop QA proves that KAG works, I am still wondering whether KAG really works in real-world scenarios, since building accurate domain-specific KG is very costly. Purely employing the exisiting information extraction methods (e.g., various tools for OpenIE) cannot output a KG with 100% correct and reliable knowledge, which may lead to hallusiciasion as well. Maybe the KAG work (Arxiv 2024) is a good example to start this research, since it successfully applied KAG to two professional knowledge Q&A tasks of the industrial downstream tasks in the Ant Group.
  • The keypoint of Graph RAG/KAG should be the wiseful integration of the benefits of unstructured information and structured information.
  1. Knowledge Graph RAG Query Engine [Link]
  2. GraphRAG: Unlocking LLM discovery on narrative private data (Feb 2024, Microsoft) [Link]
  3. Going Meta - Ep 22: RAG with Knowledge Graphs (from Neo4j) [YouTube]
  4. A very clear blog (in Chinese) to demonstrate Graph RAG: 7 种查询策略教你用好 Graph RAG 探索知识图谱 [Blog], which also provides valuable references:
  • Graph RAG LlamaIndex Workshop：[Blog]
  • Knowledge Graph Index：[Link]
  • Knowledge Graph Query Engine：[Link]
  • Knowledge Graph RAG Query Engine：[Link]
  • Custom Retriever combining KG Index and VectorStore Index：[Link]
  • LlamaIndex Webinar: Graph Databases, Knowledge Graphs, and RAG with Wey (NebulaGraph)：[Link]
  • Enhancing Large Language Models with Knowledge Graphs: The Role of Graph Algorithms
  1. ActiveRAG (a new topic!) [Notes in Chinese]

📝 KG-RAG, or GraphRAG

Based on the granularity of KGs, we classify the Graph RAG works into the following two types.
1. Works that use Coarse-grained KG
* These works first construct a KG based on input documents, then query or summerize on the KG. They usually require KG construction.
* Note: The definitions of nodes and edges of "KG" vary in different works. The nodes may be entities, nouns, document metadata, or document chunk summary; while the edges may be the semantic relations bewteen the nodes, or self-defined relations such as "co-occurrence in the same document chunk" or "appeared in the same document".
2. Works that use Fine-grained KG
* These works conduct query or reasoning on predifinded external graph (usually as input). They do not require KG construction.

(1) Overview, Surveys and Tutorials

  1. LEGO-GraphRAG: Modularizing Graph-based Retrieval-Augmented Generation for Design Space Exploration (VLDB 2025) [Paper] 🌟
  2. Graph Retrieval-Augmented Generation: A Survey (Arxiv, Aug 2024; Accepted in ACM Transactions on Information Systems 2025) [Paper]
  3. Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Survey (Arxiv, Jan 2026) [Paper]
  • Screenshot 2026-01-26 at 4 45 00 PM
  1. A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Models (Arxiv, Jan 2025) [Paper]

(2) Coarse-grained KG

  1. A good work for beginners: Convert any Corpus of Text into a Graph of Knowledge [Github] 🔥🔥🔥
  2. GraphRAG (Microsoft): From Local to Global: A Graph RAG Approach to Query-Focused Summarization (Arxiv 2024) [Paper] [GitHub]
  • The main target is to solve a query focused summarization (QFS) task, rather than an explicit retrieval task. Therefore the goal of knowledge extraction and graph construction is to group the closely related entity entities, but not to build a perfect knowledge graph.
  • No reasoning or path finding, just community detection!
  • Advanced: LazyGraphRAG, which defers LLM use (not in the extraction part, but only in the query part) and dramatically increase the efficiency of answer generation. [Source] [Discussion (Chinese)]
  • GraphRAG-Local-UI/GraphRAG-Ollama-UI, an adaptation of Microsoft's GraphRAG, tailored to support local models and featuring a comprehensive interactive user interface ecosystem. [GitHub] 🔥
  1. RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval (ICLR 2024) [Paper] hybrid search on graph
  2. KGP: Knowledge Graph Prompting for Multi-Document Question Answering (AAAI 2024) [Paper] graph query based on KNN
  3. EraRAG: Efficient and Incremental Retrieval-Augmented Generation for Growing Corpora (Arxiv 2025 June) [Paper] incremental update of the graph
  4. E^2GraphRAG: Streamlining Graph-based RAG for High Efficiency and Effectiveness (Arxiv 2025 May) [Paper] efficient search on the graph
  5. GRAG: Graph Retrieval-Augmented Generation (Arxiv 2024) [Paper] text graph
  6. GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoning (Arxiv 2024) [Paper]
  7. LeanRAG: Knowledge-Graph-Based Generation with Semantic Aggregation and Hierarchical Retrieval (AAAI 2026) [Paper]
  8. TH-RAG : Topic-Based Hierarchical Knowledge Graphs for Robust Multi-hop Reasoning in Graph-based RAG Systems (ACL 2026) [Paper]

(3) Fine-grained KG

  1. ToG: Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph (ICLR 2024) [Paper] 🔥
  • Very good motivation of why using KG reasoning instead of LLM inference + graph query.
  1. ToG 2.0: Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation (Arxiv 2024) [Paper]
  • Provides more pruning strategies for ToG.
  1. RoG: Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning (ICLR 2024) [Paper] 🔥
  2. PoG: Paths-over-graph: Knowledge graph empowered large language model reasoning (WWW 2025) [Github]
  3. DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer's Disease Questions with Scientific Literature (Arxiv 2024) [Paper]
  4. HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models [Paper]
  5. HippoRAG 2: From RAG to Memory: Non-Parametric Continual Learning for Large Language Models (ICML 2025) [Paper]
  6. G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering (NeuRIPS 2024 poster) [Paper]
  7. SUGRE: Knowledge graph-augmented language models for knowledge-grounded dialogue generation (Arxiv 2023) [Paper]
  8. GNN-RAG: Graph Neural Retrieval for Efficient Large Language Model Reasoning on Knowledge Graphs (ACL 2025) [Paper]
  9. KAG: Boosting LLMs in Professional Domains via Knowledge Augmented Generation [Paper] 🔥
  • (1) LLM-friendly knowledge representation, (2) mutual-indexing between knowledge graphs and original chunks, (3) logical-form-guided hybrid reasoning engine, (4) knowledge alignment with semantic reasoning, and (5) model capability enhancement for KAG.
  • Compared with RAG, achieving a relative improvement of 19.6% on hotpotQA and 33.5% on 2wiki in terms of F1 score.
  • Applied KAG to two professional knowledge Q&A tasks of Ant Group, including E-Government Q&A and E-Health Q&A, achieving significant improvement in professionalism compared to RAG methods.
  1. Graph RAG-Tool Fusion: [Paper] [Notes (Chinese)]
  • The construction of the "tool KG" seems to be very time-consuming.
  1. KG2RAG: Knowledge Graph-Guided Retrieval Augmented Generation (NAACL 2025, From Alibaba) [[Paper](Knowledge Graph-Guided Retrieval Augmented Generation)] [Github] but document-chunk related and users need to extract the graph
  2. KG-INFUSED RAG: AUGMENTING CORPUS-BASED RAG WITH EXTERNAL KNOWLEDGE GRAPHS (Arxiv 2025) [Paper]
  • It incorporates pre-existing large-scale KGs into RAG and applies spreading activation to enhance both retrieval and generation.
  1. ReMindRAG: Low-Cost LLM-Guided Knowledge Graph Traversal for Efficient RAG (Arxiv 2025) [Paper]
  • It employs an LLM-guided graph traversal featuring node exploration, node exploitation, and, most notably, memory replay, to improve both system effectiveness and cost efficiency. Specifically, REMINDRAG memorizes traversal experience within KG edge embeddings, mirroring the way LLMs "memorize" world knowledge within their parameters, but in a train-free manner.
  1. KG-RAG: Enhancing GUI Agent Decision-Making via Knowledge Graph-Driven Retrieval-Augmented Generation (EMNLP 2025) [Paper]
  2. Knowledge Graph Retrieval-Augmented Generation for LLM-based Recommendation (ACL 2025) [Paper]
  3. Hierarchical Planning for Complex Tasks with Knowledge Graph-RAG and Symbolic Verification (ICML 2025) [Paper]
  4. Knowledge Graph Integration and Self-Verification for Comprehensive Retrieval-Augmented Generation (2024 KDD Cup CRAG Workshop) [Paper]
  5. MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilot (Web Conference 2025) [Paper]
  6. Knowledge Graph Retrieval-Augmented Generation for LLM-based Recommendation (ACL 2025) [Paper]
  7. ShieldRAG: Safeguarding Retrieval-Augmented Generation from Untrusted Knowledge Bases (AAAI 2026) [Paper]
  8. ATLASKV: AUGMENTING LLMS WITH BILLION-SCALE KNOWLEDGE GRAPHS IN 20GB VRAM (ICLR 2026) [Paper]

(3) Multi-level grained KGs or Dynamic KGs

  1. RAG-Anything: ALL-IN-ONE RAG FRAMEWORK, a comprehensive All-in-One Multimodal Document Processing RAG system built on LightRAG [Paper] [Github]
  • LightRAG: Simple and Fast Retrieval-Augmented Generation (submitted to ICLR 2025) [Open Review][GitHub] 🔥
  1. PIKE-RAG: sPecIalized KnowledgE and Rationale Augmented Generation (Microsoft, Arxiv 2025) [Paper] [Github]
  2. Relink: Constructing Query-Driven Evidence Graph On-the-Fly for GraphRAG (AAAI 2026) [Paper]

(4) Fine-grained graph to represent procedure knowledge ("how-to" graph), not declarative knowledge ("what-is" graph)

  1. InstructRAG: Leveraging Retrieval-Augmented Generation on Instruction Graphs for LLM-Based Task Planning (SIGIR 2025)

(5) Unclassified (still working on these!)

  1. Biomedical knowledge graph-enhanced prompt generation for large language models (Nov 2023) [Paper]
  • A task-agnostic Knowledge Graph-based Retrieval Augmented Generation (KG-RAG) framework by leveraging the massive biomedical KG SPOKE with LLMs such as Llama-2-13b, GPT-3.5-Turbo and GPT-4, to generate meaningful biomedical text rooted in established knowledge.
  1. Graphusion: A RAG Framework for Knowledge Graph Construction with a Global Perspective (Arxiv 2024-10) [Paper]
  2. LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering (Arxiv 2024-10) [Paper] [Github]
  3. KRAGEN: a knowledge graph-enhanced RAG framework for biomedical problem solving using large language models (Bioinformatics 2024) [Paper]
  4. MedRAG: Enhancing Retrieval-augmented Generation with Knowledge Graph-Elicited Reasoning for Healthcare Copilot (WWW 2025 poster) [Paper]
  5. Knowledge Graph-Driven Retrieval-Augmented Generation: Integrating Deepseek-R1 with Weaviate for Advanced Chatbot Applications (Arxiv 2025) [Paper]
  6. A Systematic Exploration of Knowledge Graph Alignment with Large Language Models in Retrieval Augmented Generation (AAAI 2025) [Paper] ⭐
  7. Hierarchical Planning for Complex Tasks with Knowledge Graph-RAG and Symbolic Verification (ICML 2025) [Paper]
  8. Advancing Feature Extraction in Healthcare through the Integration of Knowledge Graphs and Large Language Models (AAAI 2025) [Paper]
  9. SimGRAG: Leveraging Similar Subgraphs for Knowledge Graphs Driven Retrieval-Augmented Generation (ACL 2025)
  10. FRAG: A Flexible Modular Framework for Retrieval-Augmented Generation based on Knowledge Graphs (ACL 2025)
  11. Distill-SynthKG: Distilling Knowledge Graph Synthesis Workflow for Improved Coverage and Efficiency (submitted to ACL 2025) [Paper]
  12. RD-P: A Trustworthy Retrieval-Augmented Prompter with Knowledge Graphs for LLMs (CIKM 2024) [Paper]
  13. HyKGE: A Hypothesis Knowledge Graph Enhanced RAG Framework for Accurate and Reliable Medical LLMs Responses (ACL 2025)
  14. GNN-RAG: Graph Neural Retrieval for Efficient Large Language Model Reasoning on Knowledge Graphs (ACL 2025, Findings) [Paper]
  15. EventRAG: Enhancing LLM Generation with Event Knowledge Graphs (ACL 2025, Findings) [Paper]
  16. OG-RAG: Ontology-Grounded Retrieval-Augmented Generation For Large Language Models (Arxiv 2024, Microsoft) [Paper] [Github]
  17. HybGRAG: Hybrid Retrieval-Augmented Generation on Textual and Relational Knowledge Bases (Arxiv 2024, Amazon) [Paper]

📊 RAG benchmarks

  1. Graph-based Approaches and Functionalities in Retrieval-Augmented Generation: A Comprehensive Survey (Arxiv, Jan 2026) [Paper]
  • It group the open-source benchmark datasets for Graph RAG into three representative families (however, the third one is also very close to the fist one)
  • Screen Shot 2026-01-29 at 10 12 26 PM

Fact Retrieval: Note: Most of the works use multi-hop QA dataset for evaluation.

  1. HotPotQA
  2. MultiHop-RAG
  3. MT-Bench

Query Focused Summerization (QFS):

  1. Document Summarization with Latent Queries (TACL 2022) [Paper]