This academic article is a thorough review of over 300 articles that focus on the use of Knowledge Graphs (KGs) in Multi-modal Learning. It discusses two main aspects: KG-driven Multi-Modal (KG4MM) learning, where KGs support multi-modal tasks, and Multi-Modal Knowledge Graph (MM4KG), which extends KG studies into the MMKG realm. The article also discusses current challenges and identifies emerging trends in Large Language Modeling and Multi-modal Pre-training strategies. The survey aims to provide a comprehensive reference for researchers involved in KG and multi-modal learning research.
Publication date: 9 Feb 2024
Project Page: https://github.com/zjukg/KG-MM-Survey
Paper: https://arxiv.org/pdf/2402.05391
