Computers and Society
New submissions
[ showing up to 2000 entries per page: fewer | more ]
New submissions for Fri, 10 May 24
- [1] arXiv:2405.05382 [pdf, other]
-
Title: DrawL: Understanding the Effects of Non-Mainstream Dialects in Prompted Image GenerationComments: 12 pages, 3 figures in main text, 2 tables in main text, 4 figures in appendix, 7 tables in appendixSubjects: Computers and Society (cs.CY)
Text-to-image models are now easy to use and ubiquitous. However, prior work has found that they are prone to recapitulating harmful Western stereotypes. For example, requesting that a model generate an "African person and their house," may produce a person standing next to a straw hut. In this example, the word "African" is an explicit descriptor of the person that the prompt is seeking to depict. Here, we examine whether implicit markers, such as dialect, can also affect the portrayal of people in text-to-image outputs. We pair prompts in Mainstream American English with counterfactuals that express grammatical constructions found in dialects correlated with historically marginalized groups. We find that through minimal, syntax-only changes to prompts, we can systematically shift the skin tone and gender of people in the generated images. We conclude with a discussion of whether dialectic distribution shifts like this are harmful or are expected, possibly even desirable, model behavior.
- [2] arXiv:2405.05420 [pdf, ps, other]
-
Title: The Power of Absence: Thinking with Archival Theory in Algorithmic DesignComments: 16 pages, to be published in the 2024 ACM Conference on Designing Interactive Systems (DIS '24)Subjects: Computers and Society (cs.CY); Human-Computer Interaction (cs.HC)
This paper explores the value of archival theory as a means of grappling with bias in algorithmic design. Rather than seek to mitigate biases perpetuated by datasets and algorithmic systems, archival theory offers a reframing of bias itself. Drawing on a range of archival theory from the fields of history, literary and cultural studies, Black studies, and feminist STS, we propose absence-as power, presence, and productive-as a concept that might more securely anchor investigations into the causes of algorithmic bias, and that can prompt more capacious, creative, and joyful future work. This essay, in turn, can intervene into the technical as well as the social, historical, and political structures that serve as sources of bias.
- [3] arXiv:2405.05596 [pdf, other]
-
Title: Measuring Strategization in Recommendation: Users Adapt Their Behavior to Shape Future ContentSubjects: Computers and Society (cs.CY); Human-Computer Interaction (cs.HC); Information Retrieval (cs.IR); Machine Learning (cs.LG); Methodology (stat.ME)
Most modern recommendation algorithms are data-driven: they generate personalized recommendations by observing users' past behaviors. A common assumption in recommendation is that how a user interacts with a piece of content (e.g., whether they choose to "like" it) is a reflection of the content, but not of the algorithm that generated it. Although this assumption is convenient, it fails to capture user strategization: that users may attempt to shape their future recommendations by adapting their behavior to the recommendation algorithm. In this work, we test for user strategization by conducting a lab experiment and survey. To capture strategization, we adopt a model in which strategic users select their engagement behavior based not only on the content, but also on how their behavior affects downstream recommendations. Using a custom music player that we built, we study how users respond to different information about their recommendation algorithm as well as to different incentives about how their actions affect downstream outcomes. We find strong evidence of strategization across outcome metrics, including participants' dwell time and use of "likes." For example, participants who are told that the algorithm mainly pays attention to "likes" and "dislikes" use those functions 1.9x more than participants told that the algorithm mainly pays attention to dwell time. A close analysis of participant behavior (e.g., in response to our incentive conditions) rules out experimenter demand as the main driver of these trends. Further, in our post-experiment survey, nearly half of participants self-report strategizing "in the wild," with some stating that they ignore content they actually like to avoid over-recommendation of that content in the future. Together, our findings suggest that user strategization is common and that platforms cannot ignore the effect of their algorithms on user behavior.
Cross-lists for Fri, 10 May 24
- [4] arXiv:2405.05347 (cross-list from cs.SE) [pdf, other]
-
Title: Benchmarking Educational Program RepairComments: 15 pages, 2 figures, 3 tables. Non-archival report presented at the NeurIPS'23 Workshop on Generative AI for Education (GAIED)Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computers and Society (cs.CY)
The emergence of large language models (LLMs) has sparked enormous interest due to their potential application across a range of educational tasks. For example, recent work in programming education has used LLMs to generate learning resources, improve error messages, and provide feedback on code. However, one factor that limits progress within the field is that much of the research uses bespoke datasets and different evaluation metrics, making direct comparisons between results unreliable. Thus, there is a pressing need for standardization and benchmarks that facilitate the equitable comparison of competing approaches. One task where LLMs show great promise is program repair, which can be used to provide debugging support and next-step hints to students. In this article, we propose a novel educational program repair benchmark. We curate two high-quality publicly available programming datasets, present a unified evaluation procedure introducing a novel evaluation metric rouge@k for approximating the quality of repairs, and evaluate a set of five recent models to establish baseline performance.
- [5] arXiv:2405.05378 (cross-list from cs.CL) [pdf, other]
-
Title: "They are uncultured": Unveiling Covert Harms and Social Threats in LLM Generated ConversationsAuthors: Preetam Prabhu Srikar Dammu, Hayoung Jung, Anjali Singh, Monojit Choudhury, Tanushree MitraSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
Large language models (LLMs) have emerged as an integral part of modern societies, powering user-facing applications such as personal assistants and enterprise applications like recruitment tools. Despite their utility, research indicates that LLMs perpetuate systemic biases. Yet, prior works on LLM harms predominantly focus on Western concepts like race and gender, often overlooking cultural concepts from other parts of the world. Additionally, these studies typically investigate "harm" as a singular dimension, ignoring the various and subtle forms in which harms manifest. To address this gap, we introduce the Covert Harms and Social Threats (CHAST), a set of seven metrics grounded in social science literature. We utilize evaluation models aligned with human assessments to examine the presence of covert harms in LLM-generated conversations, particularly in the context of recruitment. Our experiments reveal that seven out of the eight LLMs included in this study generated conversations riddled with CHAST, characterized by malign views expressed in seemingly neutral language unlikely to be detected by existing methods. Notably, these LLMs manifested more extreme views and opinions when dealing with non-Western concepts like caste, compared to Western ones such as race.
- [6] arXiv:2405.05758 (cross-list from cs.HC) [pdf, other]
-
Title: Exploring the Potential of Human-LLM Synergy in Advancing Qualitative Analysis: A Case Study on Mental-Illness StigmaComments: 55 pagesSubjects: Human-Computer Interaction (cs.HC); Computation and Language (cs.CL); Computers and Society (cs.CY)
Qualitative analysis is a challenging, yet crucial aspect of advancing research in the field of Human-Computer Interaction (HCI). Recent studies show that large language models (LLMs) can perform qualitative coding within existing schemes, but their potential for collaborative human-LLM discovery and new insight generation in qualitative analysis is still underexplored. To bridge this gap and advance qualitative analysis by harnessing the power of LLMs, we propose CHALET, a novel methodology that leverages the human-LLM collaboration paradigm to facilitate conceptualization and empower qualitative research. The CHALET approach involves LLM-supported data collection, performing both human and LLM deductive coding to identify disagreements, and performing collaborative inductive coding on these disagreement cases to derive new conceptual insights. We validated the effectiveness of CHALET through its application to the attribution model of mental-illness stigma, uncovering implicit stigmatization themes on cognitive, emotional and behavioral dimensions. We discuss the implications for future research, methodology, and the transdisciplinary opportunities CHALET presents for the HCI community and beyond.
- [7] arXiv:2405.05809 (cross-list from cs.LG) [pdf, ps, other]
-
Title: Aequitas Flow: Streamlining Fair ML ExperimentationAuthors: Sérgio Jesus, Pedro Saleiro, Inês Oliveira e Silva, Beatriz M. Jorge, Rita P. Ribeiro, João Gama, Pedro Bizarro, Rayid GhaniSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
Aequitas Flow is an open-source framework for end-to-end Fair Machine Learning (ML) experimentation in Python. This package fills the existing integration gaps in other Fair ML packages of complete and accessible experimentation. It provides a pipeline for fairness-aware model training, hyperparameter optimization, and evaluation, enabling rapid and simple experiments and result analysis. Aimed at ML practitioners and researchers, the framework offers implementations of methods, datasets, metrics, and standard interfaces for these components to improve extensibility. By facilitating the development of fair ML practices, Aequitas Flow seeks to enhance the adoption of these concepts in AI technologies.
- [8] arXiv:2405.05860 (cross-list from cs.LG) [pdf, other]
-
Title: The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human LabelsSubjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Computers and Society (cs.CY)
Longstanding data labeling practices in machine learning involve collecting and aggregating labels from multiple annotators. But what should we do when annotators disagree? Though annotator disagreement has long been seen as a problem to minimize, new perspectivist approaches challenge this assumption by treating disagreement as a valuable source of information. In this position paper, we examine practices and assumptions surrounding the causes of disagreement--some challenged by perspectivist approaches, and some that remain to be addressed--as well as practical and normative challenges for work operating under these assumptions. We conclude with recommendations for the data labeling pipeline and avenues for future research engaging with subjectivity and disagreement.
Replacements for Fri, 10 May 24
- [9] arXiv:2402.01662 (replaced) [pdf, ps, other]
-
Title: Generative Ghosts: Anticipating Benefits and Risks of AI AfterlivesComments: version 2, updated May 8, 2024 to included updated references and new case study pointers as the trend of generative ghosts acceleratesSubjects: Computers and Society (cs.CY); Artificial Intelligence (cs.AI)
- [10] arXiv:2011.08069 (replaced) [pdf, other]
-
Title: Reconciling Security and Utility in Next-Generation Epidemic Risk Mitigation SystemsAuthors: Pierfrancesco Ingo, Nichole Boufford, Ming Cheng Jiang, Rowan Lindsay, Matthew Lentz, Gilles Barthe, Manuel Gomez-Rodriguez, Bernhard Schölkopf, Deepak Garg, Peter Druschel, Aastha MehtaSubjects: Cryptography and Security (cs.CR); Computers and Society (cs.CY); Social and Information Networks (cs.SI); Populations and Evolution (q-bio.PE)
- [11] arXiv:2201.13448 (replaced) [pdf, other]
-
Title: Warmth and competence in human-agent cooperationComments: Accepted at Autonomous Agents and Multi-Agent SystemsSubjects: Human-Computer Interaction (cs.HC); Computers and Society (cs.CY); Machine Learning (cs.LG)
- [12] arXiv:2301.06774 (replaced) [pdf, other]
-
Title: Temporal Dynamics of Coordinated Online Behavior: Stability, Archetypes, and InfluenceAuthors: Serena Tardelli, Leonardo Nizzoli, Maurizio Tesconi, Mauro Conti, Preslav Nakov, Giovanni Da San Martino, Stefano CresciComments: Article published in PNAS 121 (20). Please, cite the published versionJournal-ref: Proceedings of the National Academy of Sciences 121 (20), e2307038121, 2024Subjects: Social and Information Networks (cs.SI); Computation and Language (cs.CL); Computers and Society (cs.CY)
- [13] arXiv:2309.12120 (replaced) [pdf, ps, other]
-
Title: Individual context-free online community health indicators fail to identify open source software sustainabilityComments: 99 pages, 34 tables, 19 figuresSubjects: Software Engineering (cs.SE); Computers and Society (cs.CY)
[ showing up to 2000 entries per page: fewer | more ]
Disable MathJax (What is MathJax?)
Links to: arXiv, form interface, find, cs, recent, 2405, contact, help (Access key information)