References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Text2Loc: 3D Point Cloud Localization from Natural Language
(Submitted on 27 Nov 2023 (v1), last revised 28 Mar 2024 (this version, v2))
Abstract: We tackle the problem of 3D point cloud localization based on a few natural linguistic descriptions and introduce a novel neural network, Text2Loc, that fully interprets the semantic relationship between points and text. Text2Loc follows a coarse-to-fine localization pipeline: text-submap global place recognition, followed by fine localization. In global place recognition, relational dynamics among each textual hint are captured in a hierarchical transformer with max-pooling (HTM), whereas a balance between positive and negative pairs is maintained using text-submap contrastive learning. Moreover, we propose a novel matching-free fine localization method to further refine the location predictions, which completely removes the need for complicated text-instance matching and is lighter, faster, and more accurate than previous methods. Extensive experiments show that Text2Loc improves the localization accuracy by up to $2\times$ over the state-of-the-art on the KITTI360Pose dataset. Our project page is publicly available at \url{this https URL}.
Submission history
From: Yan Xia [view email][v1] Mon, 27 Nov 2023 16:23:01 GMT (8199kb,D)
[v2] Thu, 28 Mar 2024 09:31:05 GMT (14375kb,D)
Link back to: arXiv, form interface, contact.