MMKG.md

November 14, 2025 · View on GitHub

Multi-Modal Knowledge Graph (MMKG)

Surveys and Paper Summary

  1. Multi-Modal Knowledge Graph Construction and Application: A Survey (Arxiv 2022, from Prof Yanghua Xiao) [Paper] [Notes]'
  2. What Is a Multi-Modal Knowledge Graph: A Survey (Big Data Research, 2023) [Paper]
  3. A Survey on Multimodal Knowledge Graphs: Construction, Completion and Applications (Mathematics 2023) [Paper]
  4. Awesome-Multimodal-Knowledge-Graph [GitHub]
  5. A collection of Multimoal KG Resources [GitHub]
  6. Knowledge Graphs Meet Multi-Modal Learning: A Comprehensive Survey (Arxiv, Feb 2024) [Paper] [GitHub (KG-MM-Survey)]

General Papers

  1. MMEKG: Multi-modal Event Knowledge Graph towards Universal Representation across Modalities (ACL 2022) [Paper]
  2. Multimodal Analytics for Real-world News using Measures of Cross-modal Entity Consistency (ICMR 20202) [Paper]
  • The authors work on two datasets TamperedNews & News400v (updated in IJMIR 2021) [Link]
  1. Toward a General Framework for Multimodal Big Data Analysis (Big Data 2022) [Paper]
  • Multiple data sources are first integrated into a unified knowledge graph (KG). Different modalities of data are addressed by specifying ad hoc views on the KG and producing a rewriting of the graph containing merely the data to be processed. Graph traversal and rule extraction are this way boosted.
  1. Multimodal Reasoning with Multimodal Knowledge Graphs (ACL 2024) [Paper] [Notes]
  2. Let Modalities Teach Each Other: Modal-Collaborative Knowledge Extraction and Fusion for Multimodal Knowledge Graph Completion (NAACL 2025)
  3. Adaptive Modality Interaction Transformer for Multimodal Knowledge Graph Completion (TKDD 2025) [Paper]
  4. VaLik: Aligning Vision to Language: Annotation-Free Multimodal Knowledge Graph Construction for Enhanced LLMs Reasoning (ICCV 2025) [Paper] 🔥
  • Apply cross-modal similarity recalibration to generate text descriptions and enrich the KG.
  • Training free, also reduces halluciations.
  1. VAT-KG: KNOWLEDGE-INTENSIVE MULTIMODAL KNOWLEDGE GRAPH DATASET FOR RETRIEVALAUGMENTED GENERATION (Arxiv 2025) [Paper] 🔥
  2. U-MERE: Unconstrained Multimodal Entity and Relation Extraction with Collaborative Modeling and Order-Sensitive Optimization (MM 2025) [Paper]
  3. Synergizing Multimodal Temporal Knowledge Graphs and Large Language Models for Social Relation Recognition (EMNLP 2025) 🔥

Multimodal entity linking (MEL)

  1. Multi-Grained Multimodal Interaction Network for Entity Linking (KDD 2023) [Paper]
  2. A Multimodal Entity Linking Approach Incorporating Topic Concepts (CISAI 2022) [Paper]
  3. Multimodal entity linking for tweets (HAL Open Research 2020) [Paper]
  4. Visual Entity Linking via Multi-modal Learning (Data Intelligence 2022) [Paper]
  5. Multimodal Entity Tagging with Multimodal Knowledge Base (Arxiv 2022) [Paper]
  6. Generative Multimodal Entity Linking (Arxiv 2023) [Paper] 🔥
  7. Integrating Large Language Models in Multimodal Entity Linking: A Novel Two-Level Reflection Framework (submitted to ACL 2023, open review) [Paper] 🔥
  8. Explicit-Implicit Entity Alignment Method in Multi-modal Knowledge Graphs (KDD 2025)

Benchmarks, Datasets and Evaluation

  1. (MEL Dataset) WikiDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types (ACL 2022) [Paper]
  2. (MEL Dataset) Multimodal Entity Linking: A New Dataset and A Baseline (MM 2021) [Paper]
  3. (MEL Dataset) Building a multimodal entity linking dataset from tweets (HAL Open Research 2020) [Paper]

MMKG for LLM

  1. Endowing Language Models with Multimodal Knowledge Graph Representations (Arxiv 2022) [Paper]