Abstract
Context: Software testing is a very relevant step in quality assurance, but developers frequently overlook it. We pursued testing automation to minimize the impact of missing test cases in a software project. Problem: However, for Python programs, there are not many tools able to fully automate the generation of unit test sets, and the one available demands studies to provide evidence of the quality of the generated test set. Solution: This work aims to evaluate the quality of different unit test generation algorithms for Python, implemented in a tool named Pynguin. Method: In the analysis of the selected programs, the Pynguin test generation tool is executed with each of its algorithms, including random, as a way to generate complete unit test sets. Then, we evaluate each generated test set's efficacy, efficiency, and cost. We use four different fault models, implemented by four mutation testing tools, to measure efficacy. We use line and branch coverage to measure efficiency, the number of test cases, and test set execution time to measure cost. Summary of Results: We identified that RANDOM test set performed worst concerning all evaluated aspects, DYNAMOSA and MOSA, the two algorithms that generate the best test sets regarding efficacy, efficiency, and cost. By combining all Pynguin smart algorithms (DYNAMOSA, MIO, MOSA, WHOLE-SUITE), the resultant test set overcomes the individual test sets efficiency by around 1%, for coverage and efficacy by 4.5% on average, concerning previous mutation score, at a reasonable cost, without a test set minimization.

Abstract
Context: Mock objects are commonly used in unit tests to isolate a class from its dependencies by substituting a dependency class instead of the original. Problem: However, in addition to the use of mock objects in the vast majority of OO applications, there are some discrepancies in their use. Solution: This work aims to present a case study of an industry application regarding the use of mock objects. The application in question is part of a private flight management system. IS Theory: We conceived this work based on the General Systems Theory, specifically to evaluate a microservicebased system’s use of testing doubles during testing automation. Method: In the analyzed application, mock object implementations are highly related to the system design. For example, the number of dependencies in a production class is directly related to the number of mocks in the respective test class. As a consequence, poor design choices are harmful to intrinsic quality factors such as the maintainability of the application tests. Our study uses metrics to analyze this practice. The research will consider factors of this system, aiming mainly to contribute to the improvement of test doubles by the test team. Summary of Results: Application of simulated objects in a similar system indicates that developers are making immature use of the technique, possibly due to system design problems. This study concludes that theory and practice are misaligned. Contributions and Impact in the IS area: The main contribution is to indicate points to improving software testing practice by using mocks correctly.

Abstract
With the massive adoption of mobile devices, it became more mandatory for developers to provide high-quality applications. Nowadays, mobile devices are used for different purposes: entertainment, shopping, banking, and communication. Moreover, mobile devices can communicate and exchange information with various IoT devices distributed across the city. However, mobile application testing has different challenges when compared to other types of applications (i.e., desktop and client-server applications). First, we must consider mobile devices' different characteristics and limitations, such as connectivity, screen size, density, sensors, and limited battery. Second, there is a wide range of mobile devices from diverse vendors and models. Hence, there is a need to consider different device configurations to reduce compatibility issues that may occur in a high-fragmented ecosystem. In this case, several tools and services with various features and business models aim to run tests on multiple devices. In this practical experience report, we present the initial results of implementing a testing tool/service at Von Braun Labs to support the execution of tests across multiple Android devices. The stakeholders stated the need to (i) execute the tests on physical devices; and (ii) the tool/service must support tests that interact with a specialized IoT device. We start the study by comparing different tools/services to select the most suitable one for Von Braun Labs. We propose a comparison framework to help evaluate six tools/services based on their technical, usability, and customization features. Then, we present a case study with an app from Von Braun Labs to validate the selected testing environment. Finally, we discuss the lessons learned, contributions, and future directions, pinpointing the need for a testing process since the beginning of the development project and the importance of lessening the gap between academia and industry.

Abstract
Poorly executed requirements engineering activities profoundly affect the deliverables’ quality and project’s budget and schedule. High-quality requirements reuse through requirement patterns has been widely discussed to mitigate these adverse outcomes. Requirement patterns aggregate similar applications’ behaviors and services into well-defined templates that can be reused in later specifications. The abstraction capabilities of metamodeling have shown promising results concerning the improvement of the requirement specifications’ quality and professionals’ productivity. However, there is a lack of research on requirement patterns beyond requirements engineering, even using metamodels as the underlying structure. Besides, most companies often struggle with the cost, rework, and delay effects resulting from a weak alignment between requirements and testing. In this paper, we present a novel metamodeling approach, called Software Pattern MetaModel (SoPaMM), which aligns requirements and testing through requirement patterns and test patterns. Influenced by well-established agile practices, SoPaMM describes functional requirement patterns and acceptance test patterns as user stories integrated with executable behaviors. Another novelty is the evaluation of SoPaMM’s quality properties against a metamodel quality evaluation framework. We detail the evaluation planning, discuss evaluation results, and present our study’s threats to validity. Our experience with the design and evaluation of SoPaMM is summarized as lessons learned.

Abstract

Abstract