Software Engineering

New submissions

• New submissions
• Cross-lists
• Replacements

New submissions for Fri, 10 May 24

[1]  arXiv:2405.05347 [pdf, other]

Title: Benchmarking Educational Program Repair

Authors: Charles Koutcheme, Nicola Dainese, Sami Sarsa, Juho Leinonen, Arto Hellas, Paul Denny

Comments: 15 pages, 2 figures, 3 tables. Non-archival report presented at the NeurIPS'23 Workshop on Generative AI for Education (GAIED)

Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computers and Society (cs.CY)

The emergence of large language models (LLMs) has sparked enormous interest due to their potential application across a range of educational tasks. For example, recent work in programming education has used LLMs to generate learning resources, improve error messages, and provide feedback on code. However, one factor that limits progress within the field is that much of the research uses bespoke datasets and different evaluation metrics, making direct comparisons between results unreliable. Thus, there is a pressing need for standardization and benchmarks that facilitate the equitable comparison of competing approaches. One task where LLMs show great promise is program repair, which can be used to provide debugging support and next-step hints to students. In this article, we propose a novel educational program repair benchmark. We curate two high-quality publicly available programming datasets, present a unified evaluation procedure introducing a novel evaluation metric rouge@k for approximating the quality of repairs, and evaluate a set of five recent models to establish baseline performance.

[2]  arXiv:2405.05455 [pdf, ps, other]

Title: Automated Program Repair: Emerging trends pose and expose problems for benchmarks

Authors: Joseph Renzullo, Pemma Reiter, Westley Weimer, Stephanie Forrest

Comments: 16 pages, 1 table, submitted to ACM Computing Surveys

Subjects: Software Engineering (cs.SE); Machine Learning (cs.LG)

Machine learning (ML) now pervades the field of Automated Program Repair (APR). Algorithms deploy neural machine translation and large language models (LLMs) to generate software patches, among other tasks. But, there are important differences between these applications of ML and earlier work. Evaluations and comparisons must take care to ensure that results are valid and likely to generalize. A challenge is that the most popular APR evaluation benchmarks were not designed with ML techniques in mind. This is especially true for LLMs, whose large and often poorly-disclosed training datasets may include problems on which they are evaluated.

[3]  arXiv:2405.05541 [pdf, other]

Title: CrashJS: A NodeJS Benchmark for Automated Crash Reproduction

Authors: Philip Oliver, Jens Dietrich, Craig Anslow, Michael Homer

Comments: Pre-print; Accepted to MSR 2024

Subjects: Software Engineering (cs.SE)

Software bugs often lead to software crashes, which cost US companies upwards of $2.08 trillion annually. Automated Crash Reproduction (ACR) aims to generate unit tests that successfully reproduce a crash. The goal of ACR is to aid developers with debugging, providing them with another tool to locate where a bug is in a program. The main approach ACR currently takes is to replicate a stack trace from an error thrown within a program. Currently, ACR has been developed for C, Java, and Python, but there are no tools targeting JavaScript programs. To aid the development of JavaScript ACR tools, we propose CrashJS: a benchmark dataset of 453 Node.js crashes from several sources. CrashJS includes a mix of real-world and synthesised tests, multiple projects, and different levels of complexity for both crashes and target programs.

[4]  arXiv:2405.05813 [pdf, ps, other]

Title: Revitalising Stagecraft: NLP-Driven Sentiment Analysis for Traditional Theater Revival

Authors: Saikat Samanta, Saptarshi Karmakar, Satayajay Behuria, Shibam Dutta, Soujit Das, Soumik Saha

Subjects: Software Engineering (cs.SE)

This paper explores the application of FilmFrenzy, a python based ticket booking web application, in the revival of traditional Indian theatres. Additionally, this research paper explores how NLP can be implemented to improve user experience. Through clarifying audience views and pinpointing opportunities for development, FilmFrenzy aims to promote involvement and rejuvenation in India's conventional theatre scene. The platform seeks to maintain the relevance and vitality of conventional theatres by bridging the gap between audiences and them through the incorporation of contemporary technologies, especially NLP. This research envisions a future in which technology plays a crucial part in maintaining India's rich theatrical traditions, thereby contributing to the preservation and development of cultural heritage. With sentiment analysis and natural language processing (NLP) as essential instruments for improving stagecraft, the research envisions a period when traditional theatre will still be vibrant.

Cross-lists for Fri, 10 May 24

[5]  arXiv:2405.05365 (cross-list from eess.SY) [pdf, other]

Title: Enhancing Holonic Architecture with Natural Language Processing for System of Systems

Authors: Muhammad Ashfaq, Ahmed R. Sadik, Tommi Mikkonen, Muhammad Waseem, Niko M akitalo

Comments: Preprint accepted in ICSOFT'24

Subjects: Systems and Control (eess.SY); Multiagent Systems (cs.MA); Software Engineering (cs.SE)

The complexity and dynamic nature of System of Systems (SoS) necessitate efficient communication mechanisms to ensure interoperability and collaborative functioning among constituent systems, termed holons. This paper proposes an innovative approach to enhance holon communication within SoS through the integration of Conversational Generative Intelligence (CGI) techniques. Our approach leverages advancements in CGI, specifically Large Language Models (LLMs), to enable holons to understand and act on natural language instructions. This fosters more intuitive human-holon interactions, improving social intelligence and ultimately leading to better coordination among diverse systems. This position paper outlines a conceptual framework for CGI-enhanced holon interaction, discusses the potential impact on SoS adaptability, usability and efficiency, and sets the stage for future exploration and prototype implementation.

[6]  arXiv:2405.05546 (cross-list from cs.LO) [pdf, other]

Title: Data reification in a concurrent rely-guarantee algebra

Authors: Larissa A. Meinicke, Ian J. Hayes, Cliff B. Jones

Subjects: Logic in Computer Science (cs.LO); Software Engineering (cs.SE)

Specifications of significant systems can be made short and perspicuous by using abstract data types; data reification can provide a clear, stepwise, development history of programs that use more efficient concrete representations. Data reification (or "refinement") techniques for sequential programs are well established. This paper applies these ideas to concurrency, in particular, an algebraic theory supporting rely-guarantee reasoning about concurrency. A concurrent version of the Galler-Fischer equivalence relation data structure is used as an example.

Replacements for Fri, 10 May 24

[7]  arXiv:2306.07285 (replaced) [pdf, other]

Title: TransCoder: Towards Unified Transferable Code Representation Learning Inspired by Human Skills

Authors: Qiushi Sun, Nuo Chen, Jianing Wang, Xiang Li, Ming Gao

Comments: Accepted by LREC-COLING 2024

Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI)

[8]  arXiv:2309.12120 (replaced) [pdf, ps, other]

Title: Individual context-free online community health indicators fail to identify open source software sustainability

Authors: Yo Yehudi, Carole Goble, Caroline Jay

Comments: 99 pages, 34 tables, 19 figures

Subjects: Software Engineering (cs.SE); Computers and Society (cs.CY)

[9]  arXiv:2311.16396 (replaced) [pdf, other]

Title: Toward Effective Secure Code Reviews: An Empirical Study of Security-Related Coding Weaknesses

Authors: Wachiraphan Charoenwet, Patanamon Thongtanunam, Van-Thuan Pham, Christoph Treude

Subjects: Software Engineering (cs.SE)

[10]  arXiv:2312.01639 (replaced) [pdf, other]

Title: On the Effectiveness of Large Language Models in Domain-Specific Code Generation

Authors: Yalan Lin, Meng Chen, Yuhan Hu, Hongyu Zhang, Chengcheng Wan, Zhao Wei, Yong Xu, Juhong Wang, Xiaodong Gu

Comments: Preprint submitted to ACM Transactions on Software Engineering and Methodology

Subjects: Software Engineering (cs.SE)

[11]  arXiv:2405.01309 (replaced) [pdf, other]

Title: Execution-free Program Repair

Authors: Li Huang, Bertrand Meyer, Ilgiz Mustafin, Manuel Oriol

Subjects: Software Engineering (cs.SE)

[12]  arXiv:2405.02630 (replaced) [pdf, other]

Title: cuTN-QSVM: cuTensorNet-accelerated Quantum Support Vector Machine with cuQuantum SDK

Authors: Kuan-Cheng Chen, Tai-Yue Li, Yun-Yuan Wang, Simon See, Chun-Chieh Wang, Robert Wille, Nan-Yow Chen, An-Cheng Yang, Chun-Yu Lin

Comments: 10 pages, 14 figures

Subjects: Quantum Physics (quant-ph); Distributed, Parallel, and Cluster Computing (cs.DC); Software Engineering (cs.SE)

• New submissions
• Cross-lists
• Replacements