Current browse context:
cs.CV
Change to browse by:
References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: Benchmarking the Fairness of Image Upsampling Methods
(Submitted on 24 Jan 2024 (v1), last revised 29 Apr 2024 (this version, v3))
Abstract: Recent years have witnessed a rapid development of deep generative models for creating synthetic media, such as images and videos. While the practical applications of these models in everyday tasks are enticing, it is crucial to assess the inherent risks regarding their fairness. In this work, we introduce a comprehensive framework for benchmarking the performance and fairness of conditional generative models. We develop a set of metrics$\unicode{x2013}$inspired by their supervised fairness counterparts$\unicode{x2013}$to evaluate the models on their fairness and diversity. Focusing on the specific application of image upsampling, we create a benchmark covering a wide variety of modern upsampling methods. As part of the benchmark, we introduce UnfairFace, a subset of FairFace that replicates the racial distribution of common large-scale face datasets. Our empirical study highlights the importance of using an unbiased training set and reveals variations in how the algorithms respond to dataset imbalances. Alarmingly, we find that none of the considered methods produces statistically fair and diverse results. All experiments can be reproduced using our provided repository.
Submission history
From: Mike Laszkiewicz [view email][v1] Wed, 24 Jan 2024 16:13:26 GMT (13326kb,D)
[v2] Fri, 26 Apr 2024 11:50:10 GMT (15100kb,D)
[v3] Mon, 29 Apr 2024 12:39:23 GMT (15114kb,D)
Link back to: arXiv, form interface, contact.