Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Fix-A-Step: Semi-supervised Learning from Uncurated Unlabeled Data
(Submitted on 25 Aug 2022 (v1), last revised 25 May 2023 (this version, v3))
Abstract: Semi-supervised learning (SSL) promises improved accuracy compared to training classifiers on small labeled datasets by also training on many unlabeled images. In real applications like medical imaging, unlabeled data will be collected for expediency and thus uncurated: possibly different from the labeled set in classes or features. Unfortunately, modern deep SSL often makes accuracy worse when given uncurated unlabeled data. Recent complex remedies try to detect out-of-distribution unlabeled images and then discard or downweight them. Instead, we introduce Fix-A-Step, a simpler procedure that views all uncurated unlabeled images as potentially helpful. Our first insight is that even uncurated images can yield useful augmentations of labeled data. Second, we modify gradient descent updates to prevent optimizing a multi-task SSL loss from hurting labeled-set accuracy. Fix-A-Step can repair many common deep SSL methods, improving accuracy on CIFAR benchmarks across all tested methods and levels of artificial class mismatch. On a new medical SSL benchmark called Heart2Heart, Fix-A-Step can learn from 353,500 truly uncurated ultrasound images to deliver gains that generalize across hospitals.
Submission history
From: Zhe Huang [view email][v1] Thu, 25 Aug 2022 04:52:21 GMT (283kb,D)
[v2] Thu, 2 Mar 2023 17:27:13 GMT (6939kb,D)
[v3] Thu, 25 May 2023 18:23:40 GMT (6939kb,D)
Link back to: arXiv, form interface, contact.