Place Recognition Meet multiple Modalities: A Comprehensive Review, Current Challenges and Future Directions

Zhenyu Li, Tianyi Shang, Pengjie Xu, Zhaojun Deng

Abstract

Place recognition is a cornerstone of vehicle navigation and mapping, which is pivotal in enabling systems to determine whether a location has been previously visited. This capability is critical for tasks such as loop closure in Simultaneous Localization and Mapping (SLAM) and long-term navigation under varying environmental conditions. This survey comprehensively reviews recent advancements in place recognition, emphasizing three representative methodological paradigms: Convolutional Neural Network (CNN)-based approaches, Transformer-based frameworks, and cross-modal strategies. We begin by elucidating the significance of place recognition within the broader context of autonomous systems. Subsequently, we trace the evolution of CNN-based methods, highlighting their contributions to robust visual descriptor learning and scalability in large-scale environments. We then examine the emerging class of Transformer-based models, which leverage self-attention mechanisms to capture global dependencies and offer improved generalization across diverse scenes. Furthermore, we discuss cross-modal approaches that integrate heterogeneous data sources such as Lidar, vision, and text description, thereby enhancing resilience to viewpoint, illumination, and seasonal variations. We also summarize standard datasets and evaluation metrics widely adopted in the literature. Finally, we identify current research challenges and outline prospective directions, including domain adaptation, real-time performance, and lifelong learning, to inspire future advancements in this domain.

Survey Overview

This paper provides a comprehensive review of recent advancements in place recognition, focusing on three key methodological paradigms:

1. CNN-based Approaches
2. Transformer-based Frameworks
3. Cross-modal Strategies

1. Introduction

Significance in Autonomous Systems

Place recognition plays a pivotal role in:

• Autonomous vehicle navigation
• Large-scale environment mapping
• Robust localization under changing conditions

2. Methodological Evolution

2.1 CNN-based Approaches

Key Contributions:

• Robust visual descriptor learning
• Scalability in large-scale environments
• Evolution from traditional features to deep learning

2.2 Transformer-based Models

Advancements:

• Self-attention mechanisms capturing global dependencies
• Improved generalization across diverse scenes
• Handling of long-range spatial relationships

2.3 Cross-modal Strategies

Innovations:

• Integration of heterogeneous data sources:

  • Lidar point clouds
  • Visual information
  • Text descriptions

• Enhanced resilience to:

  • Viewpoint variations
  • Illumination changes
  • Seasonal transitions

  

3. Challenges & Future Directions

Current Research Challenges

• Domain adaptation across environments
• Real-time performance requirements
• Lifelong learning capabilities

Prospective Research Directions

1. Adaptive Systems

   • Cross-domain generalization
   • Continuous learning frameworks

2. Efficiency Optimization

   • Computational efficiency improvements
   • Memory-constrained implementations

3. Advanced Fusion Techniques

   • Multi-modal integration
   • Temporal consistency methods

4. All the methods are listed below:

Title	First Author	Venue	Github	Bibtex
Gsv-cities: Toward appropriate supervised visual place recognition	Amar Ali-bey	Neurocomputing 2022	Github	BibTex
Mixvpr: Feature mixing for visual place recognition	Amar Ali-bey	WACV 2023	Github	BibTex
BoQ: A place is worth a bag of learnable queries	Amar Ali-bey	CVPR 2024	Github	BibTex
NetVLAD: CNN Architecture for Weakly Supervised Place Recognition	Relja Arandjelovic	CVPR 2016	Github	BibTex
AttDLNet: Attention-based Deep Network for 3D LiDAR Place Recognition	Tiago Barros	Robot 2022	Github	BibTex
Place recognition survey: An update on deep learning approaches	Tiago Barros	arXiv		BibTex
Rethinking visual geo-localization for large-scale applications	Gabriele Berton	CVPR 2022	Github	BibTex
Eigenplaces: Training viewpoint robust models for visual place recognition	Gabriele Berton	ICCV 2023	Github	BibTex
Unifying deep local and global features for image search	Bingyi Cao	ECCV 2020	Github	BibTex
Emerging properties in self-supervised vision transformers	Mathilde Caron	ICCV 2021		BibTex
Lcdnet: Deep loop closure detection and point cloud registration for lidar slam	Daniele Cattaneo	TRO 2022	Github	BibTex
SpoxelNet: Spherical voxel-based deep place recognition for 3D point clouds of crowded indoor spaces	Min Young Chang	IROS 2020		BibTex
Convolutional neural network-based place recognition	Zetao Chen	arXiv		BibTex
FAB-MAP: Probabilistic localization and mapping in the space of appearance	Mark Cummins	IJRR 2008	Github	BibTex
A solution to the simultaneous localization and map building (SLAM) problem	MWM Gamini Dissanayake	TRO 2001		BibTex
Dh3d: Deep hierarchical 3d descriptors for robust large-scale 6dof relocalization	Juan Du	ECCV 2020	Github	BibTex
Direct sparse odometry	Jakob Engel	TPAMI 2017	Github	BibTex
Svt-net: Super light-weight sparse voxel transformer for large scale place recognition	Zhaoxin Fan	AAAI 2022	Github	BibTex
Adaptive mobile robot navigation and mapping	HJS Feder	IJRR 1999		BibTex
Toward object-based place recognition in dense rgb-d maps	Dorian Gálvez-López	TRO 2012		BibTex
Bags of binary words for fast place recognition in image sequences	Dorian Gálvez-López	TRO 2012		BibTex
Revisit Anything: Visual Place Recognition via Image Segment Retrieval	Kartik Garg	ECCV 2024	Github	BibTex
Self-supervising ffne-grained region similarities for large-scale image localization	Yixiao Ge	ECCV 2020	Github	BibTex
FAB-MAP+ RatSLAM: Appearance-based SLAM for multiple times of day	AJ Glover	ICRA 2010		BibTex
The perfect match: 3d point cloud matching with smoothed densities	Zan Gojcic	CVPR 2019	Github	BibTex
Salsa: Swift adaptive lightweight self-attention for enhanced lidar place recognition	Raktim Gautam Goswami	RAL 2024	Github	BibTex
Indoor localization improved by spatial context—A survey	Fuqiang Gu	ACM Computing Surveys		BibTex
Recent trends in task and motion planning for robotics: A survey	Huihui Guo	Computing Surveys		BibTex
Visual place recognition using HMM sequence matching	Peter Hansen	IROS 2014		BibTex
Patch-netvlad: Multi-scale fusion of locally-global descriptors for place recognition	Stephen Hausler	CVPR 2021	Github	BibTex
Pair-vpr: Place-aware pre-training and contrastive pair classiffcation for visual place recognition with vision transformers	Stephen Hausler	RAL 2025	Github	BibTex
Hitpr: Hierarchical transformer for place recognition in point cloud	Zhixing Hou	ICRA 2022		BibTex
Progeo: Generating prompts through image-text contrastive learning for visual geo-localization	Jingqi Hu	ICANN 2024	Github	BibTex
360loc: A dataset and benchmark for omnidirectional visual localization with cross-device queries	Huajian Huang	CVPR 2024	Github	BibTex
Cross-modal and uni-modal soft-label alignment for image-text retrieval	Hailang Huang	AAAI 2024	Github	BibTex
Optimal transport aggregation for visual place recognition	Sergio Izquierdo	CVPR 2024	Github	BibTex
Learned contextual feature reweighting for image geo-localization	Hyo Jin Kim	CVPR 2017	Github	BibTex
HeLiPR: Heterogeneous LiDAR dataset for inter-LiDAR place recognition under spatiotemporal variations	Minwoo Jung	IJRR 2024	Github	BibTex
HeLiPR: Heterogeneous LiDAR dataset for inter-LiDAR place recognition under spatiotemporal variations	Minwoo Jung	IJRR 2024	Github	BibTex
Anyloc: Towards universal visual place recognition	Nikhil Keetha	RAL 2023	Github	BibTex
A holistic visual place recognition approach using lightweight cnns for signiffcant viewpoint and appearance changes	Ahmad Khaliq	TRO 2019		BibTex
Level-5 autonomous driving—Are we there yet? A review of research literature	Manzoor Ahmed Khan	ACM Computing Surveys		BibTex
Narrowing your fov with solid: Spatially organized and lightweight global descriptor for fov-constrained lidar place recognition	Hogyun Kim	RAL 2024	Github	BibTex
Text2pos: Text-to-point-cloud cross-modal localization	Manuel Kolmet	CVPR 2022	Github	BibTex
Minkloc3d: Point cloud based large-scale place recognition	Jacek Komorowski	WACV 2021	Github	BibTex
Improving point cloud based place recognition with ranking-based loss and large batch training	Jacek Komorowski	ICPR 2022	Github	BibTex
Generalized contrastive optimization of siamese networks for place recognition	María Leyva-Vallina	arXiv	Github	BibTex
Toward Robust Visual Place Recognition for Mobile Robots With an End-to-End Dark-Enhanced Net	Zhenyu Li	TII 2025	Github	BibTex
CSPFormer: A cross-spatial pyramid transformer for visual place recognition	Zhenyu Li	Neurocomputing 2024		BibTex
Feature-Level Knowledge Distillation for Place Recognition Based on Soft-Hard Labels Teaching Paradigm	Zhenyu Li	TIIS 2025	Github	BibTex
CWPFormer: Towards High-performance Visual Place Recognition for Robot with Cross-weight Attention Learning	Zhenyu Li	TAI 2025	Github	BibTex
Translo: A window-based masked point transformer framework for large-scale lidar odometry	Jiuming Liu	AAAI 2023	Github	BibTex
Stochastic attraction-repulsion embedding for large scale image localization	Liu Liu	ICCV 2019	Github	BibTex
Stochastic attraction-repulsion embedding for large scale image localization	Liu Liu	ICCV 2019	Github	BibTex
Visual place recognition: A survey	Stephanie Lowry	TRO 2015		BibTex
Unsupervised online learning of condition-invariant images for place recognition	Stephanie Lowry	ACRA 2014		BibTex
Deep homography estimation for visual place recognition	Feng Lu	AAAI 2024	Github	BibTex
SelaVPR++: Towards Seamless Adaptation of Foundation Models for Efffcient Place Recognition	Feng Lu	arXiv	Github	BibTex
Cricavpr: Cross-image correlation-aware representation learning for visual place recognition	Feng Lu	CVPR 2024	Github	BibTex
Towards seamless adaptation of pre-trained models for visual place recognition	Feng Lu	arXiv	Github	BibTex
3D point cloud-based place recognition: a survey	Kan Luo	Artiffcial Intelligence Review		BibTex
BEVPlace: Learning LiDAR-based place recognition using bird’s eye view images	Lun Luo	ICCV 2023	Github	BibTex
Seqot: A spatial–temporal transformer network for place recognition using sequential lidar data	Junyi Ma	TIE 2022	Github	BibTex
OverlapTransformer: An efffcient and yaw-angle-invariant transformer network for LiDAR-based place recognition	Junyi Ma	RAL 2022	Github	BibTex
1 year, 1000 km: The oxford robotcar dataset	Will Maddern	IJRR 2017	Dataset	BibTex
SeqSLAM: Visual route-based navigation for sunny summer days and stormy winter nights	Michael J Milford	ICRA 2012	Github	BibTex
Environment selection and hierarchical place recognition	Mahesh Mohan	ICRA 2015		BibTex
FastSLAM: A factored solution to the simultaneous localization and mapping problem	Michael Montemerlo	AAAI 2002	Github	BibTex
ORB-SLAM: A versatile and accurate monocular SLAM system	Raul Mur-Artal	TRO 2002		BibTex
A comprehensive review on autonomous navigation	Saeid Nahavandi	Computing Surveys		BibTex
The mapillary vistas dataset for semantic understanding of street scenes	Gerhard Neuhold	ICCV 2017	Dataset	BibTex
Single-view place recognition under seasonal changes	Daniel Olid	Arxiv	Github	BibTex
Dinov2: Learning robust visual features without supervision	Maxime Oquab	Arxiv	Github	BibTex
Visual place recognition using landmark distribution descriptors	Pilailuck Panphattarasap	ACCV 2016		BibTex
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation	Charles R Qi	CVPR 2017	Github	BibTex
Pointnet++: Deep hierarchical feature learning on point sets in a metric space	Charles Ruizhongtai Qi	neurips 2017	Github	BibTex
Fine-tuning CNN image retrieval with no human annotation	Filip Radenović	TPAMI 2018	Github	BibTex
Learning transferable visual models from natural language supervision	Alec Radford	ICML 2021	Github	BibTex
Vlocnet++: Deep multitask learning for semantic visual localization and odometry	Noha Radwan	RAL 2018		BibTex
Learning with average precision: Training image retrieval with a listwise loss	Jerome Revaud	ICCV 2019	Github	BibTex
Superglue: Learning feature matching with graph neural networks	Paul-Edouard Sarlin	CVPR 2020	Github	BibTex
MambaPlace: Text-to-Point-Cloud Cross-Modal Place Recognition with Attention Mamba Mechanisms	Tianyi Shang	arXiv	Github	BibTex
Text-Driven 3D Lidar Place Recognition for Autonomous Driving	Tianyi Shang	arXiv	Github	BibTex
Bridging Text and Vision: A Multi-View Text-Vision Registration Approach for Cross-Modal Place Recognition	Tianyi Shang	arXiv	Github	BibTex
Voxel-based representation learning for place recognition based on 3d point clouds	Sriram Siva	IROS 2020		BibTex
A dataset for benchmarking image-based localization	Xun Sun	CVPR 2017		BibTex
On the performance of convnet features for place recognition	Niko Sünderhauf	IROS 2015		BibTex
OpenSeqSLAM2. 0: An open source toolbox for visual place recognition under changing conditions	Ben Talbot	IROS 2018	Github	BibTex
OpenSeqSLAM2. 0: An open source toolbox for visual place recognition under changing conditions	Ben Talbot	IROS 2018	Github	BibTex
The graph SLAM algorithm with applications to large-scale mapping of urban structures	Sebastian Thrun	IJRR 2006		BibTex
24/7 place recognition by view synthesis	Akihiko Torii	CVPR 2015		BibTex
Visual place recognition with repetitive structures	Akihiko Torii	CVPR 2013		BibTex
Effovpr: Effective foundation model utilization for visual place recognition	Issar Tzachor	ICLR 2025		BibTex
Pointnetvlad: Deep point cloud based retrieval for large-scale place recognition	Mikaela Angelina Uy	CVPR 2018	Github	BibTex
LoGG3D-Net: Locally guided global descriptor learning for 3D place recognition	Kavisha Vidanapathirana	ICRA 2022	Github	BibTex
Text to point cloud localization with relation-enhanced transformer	Guangzhi Wang	AAAI 2023		BibTex
Transvpr: Transformer-based place recognition with multi-level attention aggregation	Ruotong Wang	CVPR 2022	Github	BibTex
Deepvo: Towards end-to-end visual odometry with deep recurrent convolutional neural networks	Sen Wang	ICRA 2017	Github	BibTex
Ranking-aware Continual Learning for LiDAR Place Recognition	Xufei Wang	arXiv		BibTex
Text2loc: 3d point cloud localization from natural language	Yan Xia	CVPR 2024	Github	BibTex
TransLoc3D: Point cloud based large-scale place recognition using adaptive receptive ffelds	Tian-Xing Xu	arXiv	Github	BibTex
TransVLAD: Multi-scale attention-based global descriptors for visual geo-localization	Yifan Xu	ECCV 2023	Github	BibTex
Hierarchical attention fusion for geo-localization	Liqi Yan	ICASSP 2021	Github	BibTex
Autonomous visual navigation for mobile robots: A systematic literature review	Yuri DV Yasuda	Computing Surveys		BibTex
Mrs-vpr: a multi-resolution sampling based global visual place recognition method	Peng Yin	ICRA 2019		BibTex
Spatial pyramid-enhanced NetVLAD with weighted triplet loss for place recognition	Jun Yu	TNNLS 2019		BibTex
3dmatch: Learning local geometric descriptors from rgb-d reconstructions	Andy Zeng	CVPR 2017	Github	BibTex
PCAN: 3D attention map learning using contextual information for point cloud based retrieval	Wenxiao Zhang	CVPR 2019	Github	BibTex
Lidar-based place recognition for autonomous driving: A survey	Yongjun Zhang	Computing Surveys		BibTex
Learning deep features for scene recognition using places database	Bolei Zhou	NeurIPS 2014		BibTex
Loop closure detection using local 3D deep descriptors	Youjie Zhou	RAL 2022	Github	BibTex
Ndt-transformer: Large-scale 3d point cloud localisation using the normal distribution transform representation	Zhicheng Zhou	ICRA 2021	Github	BibTex
R2former: Uniffed retrieval and reranking transformer for place recognition	Sijie Zhu	CVPR 2023	Github	BibTex
PRGS: Patch-to-Region Graph Search for Visual Place Recognition	Weiliang Zuo	Pattern Recognition 2025	Github	BibTex
A2GC: Asymmetric Aggregation with Geometric Constraints for Locally Aggregated Descriptors	Zhenyu Li	arxiv 2025	Github	BibTex
FourierPlace: A Vision-Language Localization Framework Based on Frequency Domain Representations	Tianyi Shang	IEEE RAL 2025	Github	BibTex

Cite this article:

Li, Z., Shang, T., Xu, P. et al. Place recognition meet multiple modalities: a comprehensive review, current challenges and future development. Artif Intell Rev 58, 363 (2025). https://doi.org/10.1007/s10462-025-11367-8