2024
Autores
Guerino, LR; Kuroishi, PH; Paiva, ACR; Vincenzi, AMR;
Publicação
23TH BRAZILIAN SYMPOSIUM ON SOFTWARE QUALITY, SBQS 2024
Abstract
Context: Mutation testing is a rigorous approach for assessing the quality of test suites by injecting faults (i.e., mutants) into software under test. Tools, such as CosmicRay and Mutpy, are examples of Mutation Testing tools for Python software programs. Problem: With different Python mutation testing tools, comparative analysis is lacking to evaluate their effectiveness in different usage scenarios. Furthermore, the evolution of these tools makes continuous evaluation of their functionalities and characteristics necessary. Method: In this work, we evaluate (statically and dynamically) four Python mutation testing tools, namely CosmicRay, MutPy, MutMut, and Mutatest. In static evaluation, we introduce a comparison framework, adapted from one previously applied to Java tools, and collected information from tool documentation and developer surveys. For dynamic evaluation, we use tests built based on those produced by Pynguin, which are improved through the application of Large Language Models (LLMs) and manual analyses. Then, the adequate test suites were cross-tested among different tools to evaluate their effectiveness in killing mutants each other. Results: Our findings reveal that CosmicRay offers superior functionalities and customization options for mutant generation compared to its counterparts. Although CosmicRay's performance was slightly lower than MutPy in the dynamic tests, its recent updates and active community support highlight its potential for future enhancements. Cross-examination of the test suites further shows that mutation scores varied narrowly among tools, with a slight emphasis on MutPy as the most effective mutant fault model.
2024
Autores
Silva, M; Paiva, ACR; Mendes, A;
Publicação
SOFTWARE QUALITY JOURNAL
Abstract
Software testing plays a fundamental role in software engineering, involving the systematic evaluation of software to identify defects, errors, and vulnerabilities from the early stages of the development process. Education in software testing is essential for students and professionals, as it promotes quality and favours the construction of reliable software solutions. However, motivating students to learn software testing may be a challenge. To overcome this, educators may incorporate some strategies into the teaching and learning process, such as real-world examples, interactive learning, and gamification. Gamification aims to make learning software testing more engaging for students by creating a more enjoyable experience. One approach that has proven effective is to use serious games. This paper presents a novel serious game to teach white-box testing test case design techniques, named GAMFLEW (GAMe For LEarning White-box testing). It describes the design, game mechanics, and its implementation. It also presents a preliminary evaluation experiment with students to assess the usability, learnability, and perceived problems, among other aspects. The results obtained are encouraging.
2024
Autores
Tramontana, P; Marín, B; Paiva, ACR; Mendes, A; Vos, TEJ; Amalfitano, D; Cammaerts, F; Snoeck, M; Fasolino, AR;
Publicação
2024 IEEE CONFERENCE ON SOFTWARE TESTING, VERIFICATION AND VALIDATION, ICST 2024
Abstract
Software testing is an indispensable component of software development, yet it often receives insufficient attention. The lack of a robust testing culture within computer science and informatics curricula contributes to a shortage of testing expertise in the software industry. Addressing this problem at its root -education- is paramount. In this paper, we conduct a comprehensive mapping review of software testing courses, elucidating their core attributes and shedding light on prevalent subjects and instructional methodologies. We mapped 117 courses offered by Computer Science (and related) degrees in 49 academic institutions from four Western European countries, namely Belgium, Italy, Portugal and Spain. The testing subjects were mapped against the conceptual framework provided by the ISO/IEC/IEEE 29119 standard on software testing. Among the results, the study showed that dedicated software testing courses are offered by only 39% of the analysed universities, whereas the basics of software testing are taught in at least one course at every university. The analysis of the software testing topics highlights the gaps that need to be filled in order to better align the current academic offerings with the real industry needs.
2024
Autores
Paiva, CR; Abreu, R;
Publicação
Proceedings - International Conference on Software Engineering
Abstract
[No abstract available]
2024
Autores
Tavares, P; Paiva, A; Amalfitano, D; Just, R;
Publicação
PROCEEDINGS OF THE 33RD ACM SIGSOFT INTERNATIONAL SYMPOSIUM ON SOFTWARE TESTING AND ANALYSIS, ISSTA 2024
Abstract
Mutation testing has evolved beyond academic research, is deployed in industrial and open-source settings, and is increasingly part of universities' software engineering curricula. While many mutation testing tools exist, each with different strengths and weaknesses, integrating them into educational activities and exercises remains challenging due to the tools' complexity and the need to integrate them into a development environment. Additionally, it may be desirable to use different tools so that students can explore differences, e.g.. in the types or numbers of generated mutants. Asking students to install and learn multiple tools would only compound technical complexity and likely result in unwanted differences in how and what students learn. This paper presents FRAFOL, a framework for learning mutation testing. FRAME provides a common environment for using different mutation testing tools in an educational setting.
2024
Autores
Rodrigues, J; Lopes, CT;
Publicação
METADATA AND SEMANTIC RESEARCH, MTSR 2023
Abstract
Research data management includes activities that organize and manage the life of a research project and is crucial for consistent work performance. Some activities are related to the description, which is a fundamental step, since it allows data to be properly documented and interpreted, promoting their subsequent reuse and sharing. The description is usually done through text, but other typologies can also be used, such as images, taking advantage of their potential and particular characteristics to promote description. We used a qualitative method of investigation through an exploratory case study. We conducted 16 semi-structured interviews, with researchers who have produced, described, and published research data, in order to understand how images can assume the role of metadata in data description. We found that all interviewees would like to have the possibility of describing data with images, but they consider that the publishing platforms have to be prepared for this. Most researchers were able to identify descriptors that could include images and also describe those that they consider being the greatest advantages of the project. All researchers consider that images as metadata would be a more direct gateway to the data. The issue of data description through resources other than text has never been properly investigated. The existing literature does not develop the theme, although images have had an abrupt growth in society and science. This work aims to open new paths, raise new ideas and raise awareness of new and original practices.
The access to the final selection minute is only available to applicants.
Please check the confirmation e-mail of your application to obtain the access code.