References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Segment Every Out-of-Distribution Object
(Submitted on 27 Nov 2023 (v1), last revised 28 Mar 2024 (this version, v4))
Abstract: Semantic segmentation models, while effective for in-distribution categories, face challenges in real-world deployment due to encountering out-of-distribution (OoD) objects. Detecting these OoD objects is crucial for safety-critical applications. Existing methods rely on anomaly scores, but choosing a suitable threshold for generating masks presents difficulties and can lead to fragmentation and inaccuracy. This paper introduces a method to convert anomaly \textbf{S}core \textbf{T}o segmentation \textbf{M}ask, called S2M, a simple and effective framework for OoD detection in semantic segmentation. Unlike assigning anomaly scores to pixels, S2M directly segments the entire OoD object. By transforming anomaly scores into prompts for a promptable segmentation model, S2M eliminates the need for threshold selection. Extensive experiments demonstrate that S2M outperforms the state-of-the-art by approximately 20% in IoU and 40% in mean F1 score, on average, across various benchmarks including Fishyscapes, Segment-Me-If-You-Can, and RoadAnomaly datasets.
Submission history
From: Wenjie Zhao [view email][v1] Mon, 27 Nov 2023 18:20:03 GMT (44655kb,D)
[v2] Sat, 9 Dec 2023 01:29:02 GMT (44651kb,D)
[v3] Wed, 13 Dec 2023 17:51:29 GMT (44651kb,D)
[v4] Thu, 28 Mar 2024 15:15:04 GMT (30125kb,D)
Link back to: arXiv, form interface, contact.