AnyLoc: Towards Universal Visual Place Recognition

IEEE Robotics and Automation Letters (RA-L) 2023 & ICRA 2024

1CMU, 2IIIT-Hyderabad, 3MIT, 4AIML, University of Adelaide
*Co-first authors

AnyLoc enables universal visual place recognition (VPR) anywhere, anytime and under anyview.

Abstract

VPR is vital for robot localization. To date, the most performant VPR approaches are environment- and task-specific: while they exhibit strong performance in structured environments (predominantly urban driving), their performance degrades severely in unstructured environments, rendering most approaches brittle to robust realworld deployment. In this work, we develop a universal solution to VPR – a technique that works across a broad range of structured and unstructured environments (urban, outdoors, indoors, aerial, underwater, and subterranean environments) without any re-training or finetuning. We demonstrate that general-purpose feature representations derived from off-theshelf self-supervised models with no VPR-specific training are the right substrate upon which to build such a universal VPR solution. Combining these derived features with unsupervised feature aggregation enables our suite of methods, AnyLoc, to achieve up to 4× significantly higher performance than existing approaches. We further obtain a 6% improvement in performance by characterizing the semantic properties of these features, uncovering unique domains which encapsulate datasets from similar environments. Our detailed experiments and analysis lay a foundation for building VPR solutions that may be deployed anywhere, anytime, and across anyview.

AnyLoc Performance

Video

Try out Universal VPR in action!

Simply click on any point on the query trajectory (upper left), and observe its best retrieval and similarity heatmap on the database trajectory (upper right). Use the drop-down menu to switch to a different environment. The corresponding query and the best retrieval image is visualized at the lower left and lower right respectively. Points that are identified by ⭐ on the database trajectory indicate the best retrieval. Yellow indicates higher similarity in the database trajectory (upper right).

Query Image

Retrieved Image

Qualitative Retrieval Visualizations

VPAir

Hawkins

Laurel Caverns

Vocabulary Visualizations on Gardens Point

Database Sample

Query Sample

Vocabulary Visualizations comparing DINO & DINOv2

DINO ViT-S L9 Key

DINOv2 ViT-S L10 Value

Vocabulary Visualizations across DINO & DINOv2 layers

DINO ViT-S Key

DINOv2 ViT-S Value

DINOv2 ViT-G Value

BibTeX


      @article{keetha2023anyloc,
        title={Anyloc: Towards universal visual place recognition},
        author={Keetha, Nikhil and Mishra, Avneesh and Karhade, Jay and Jatavallabhula, Krishna Murthy and Scherer, Sebastian and Krishna, Madhava and Garg, Sourav},
        journal={IEEE Robotics and Automation Letters},
        year={2023},
        publisher={IEEE}
      }