Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: GenURL: A General Framework for Unsupervised Representation Learning
(Submitted on 27 Oct 2021 (v1), revised 11 Oct 2022 (this version, v2), latest version 16 Apr 2024 (v4))
Abstract: Unsupervised representation learning (URL) that learns compact embeddings of complex data without supervision has achieved remarkable progress recently. Although the general goal of URL is similar across various scenarios, the actual algorithms differ widely in different tasks because they were separately designed according to a specific URL task. In this paper, we develop a general and efficient similarity-based URL framework called GenURL, which can adapt to various URL tasks in a unified manner. Based on the manifold assumption, we regard most URL tasks as an embedding problem that seeks an optimal low-dimensional representation for the high-dimensional data. The learning process contains two steps, data structural modeling, and low-dimensional embedding. Specifically, we provide a general method to model data structures by adaptively combining graph distances on the predefined graphs, then propose robust loss functions for the low-dimensional embedding objective. Combined with a specific pretext task, GenURL achieves state-of-the-art or competitive performance in self-supervised visual representation learning, unsupervised knowledge distillation, graph embeddings, and dimension reduction.
Submission history
From: Siyuan Li [view email][v1] Wed, 27 Oct 2021 16:24:39 GMT (9643kb,D)
[v2] Tue, 11 Oct 2022 06:24:25 GMT (8755kb,D)
[v3] Mon, 17 Oct 2022 17:23:31 GMT (8975kb,D)
[v4] Tue, 16 Apr 2024 21:52:28 GMT (15601kb,D)
Link back to: arXiv, form interface, contact.