Efficiency of Software Testing Techniques: A Controlled Experiment Replication and Network Meta-analysis
Journal Title: e-Informatica Software Engineering Journal - Year 2017, Vol 11, Issue 1
Abstract
Background. Common approaches to software verification include static testing techniques, such as code reading, and dynamic testing techniques, such as black-box and white-box testing. Objective. With the aim of gaining a better understanding of software testing techniques, a controlled experiment replication and the synthesis of previous experiments which examine the efficiency of code reading, black-box and white-box testing techniques were conducted. Method. The replication reported here is composed of four experiments in which instrumented programs were used. Participants randomly applied one of the techniques to one of the instrumented programs. The outcomes were synthesized with seven experiments using the method of network meta-analysis (NMA). Results. No significant differences in the efficiency of the techniques were observed. However, it was discovered the instrumented programs had a significant effect on the efficiency. The NMA results suggest that the black-box and white-box techniques behave alike; and the efficiency of code reading seems to be sensitive to other factors. Conclusion. Taking into account these findings, the Authors suggest that prior to carrying out software verification activities, software engineers should have a clear understanding of the software product to be verified; they can apply either black-box or white-box testing techniques as they yield similar defect detection rates.
Authors and Affiliations
Omar Gómez, Karen Cortés-Verdín, César Pardo
Keywords
Related Articles
- EP ID EP200433
- DOI 10.5277/e-Inf170104
- Views 122
- Downloads 0
Measuring Goal-Oriented Requirements Language Actor Stability
Background: Goal models describe interests, preferences, intentions, desired goals and strategies of intervening stakeholders during the early requirements engineering stage. When capturing the requirements of real-world...
Reducing the Number of Higher-order Mutants with the Aid of Data Flow
Higher-order mutants are created by injecting two or more mutations into the original program, while first-order mutants are generated by seeding single faults in the original program. Mutant generation is a key stage of...
Tool Features to Support Systematic Reviews in Software Engineering – A Cross Domain Study
Context: Previously, the authors had developed and evaluated a framework to evaluate systematic review (SR) lifecycle tools. Goal: The goal of this study was to use the experiences of researchers in other domains to furt...
Resolving Conflict and Dependency in Refactoring to a Desired Design
Refactoring is performed to improve software quality while leaving the behaviour of the system unchanged. In practice there are many opportunities for refactoring, however, due to conflicts and dependencies between refac...
Cross-Project Defect Prediction with Respect to Code Ownership Model: An Empirical Study
The paper presents an analysis of 83 versions of industrial, open-source and academic projects. We have empirically evaluated whether those project types constitute separate classes of projects with regard to defect pred...