SWFC-ART: A Cost-effective Approach for Fixed-Size-Candidate-Set Adaptive Random Testing through Small World Graphs
Authors:
Muhammad Ashfaq,
Rubing Huang,
Dave Towey,
Michael Omari,
Dmitry Yashunin,
Patrick Kwaku Kudjo,
Tao Zhang
Abstract:
Adaptive random testing (ART) improves the failure-detection effectiveness of random testing by leveraging properties of the clustering of failure-causing inputs of most faulty programs: ART uses a sampling mechanism that evenly spreads test cases within a software's input domain. The widely-used Fixed-Sized-Candidate-Set ART (FSCS-ART) sampling strategy faces a quadratic time cost, which worsens…
▽ More
Adaptive random testing (ART) improves the failure-detection effectiveness of random testing by leveraging properties of the clustering of failure-causing inputs of most faulty programs: ART uses a sampling mechanism that evenly spreads test cases within a software's input domain. The widely-used Fixed-Sized-Candidate-Set ART (FSCS-ART) sampling strategy faces a quadratic time cost, which worsens as the dimensionality of the software input domain increases. In this paper, we propose an approach based on small world graphs that can enhance the computational efficiency of FSCS-ART: SWFC-ART. To efficiently perform nearest neighbor queries for candidate test cases, SWFC-ART incrementally constructs a hierarchical navigable small world graph for previously executed, non-failure-causing test cases. Moreover, SWFC-ART has shown consistency in programs with high dimensional input domains. Our simulation and empirical studies show that SWFC-ART reduces the computational overhead of FSCS-ART from quadratic to log-linear order while maintaining the failure-detection effectiveness of FSCS-ART, and remaining consistent in high dimensional input domains. We recommend using SWFC-ART in practical software testing scenarios, where real-life programs often have high dimensional input domains and low failure rates.
△ Less
Submitted 12 May, 2021;
originally announced May 2021.
A Taxonomic Review of Adaptive Random Testing: Current Status, Classifications, and Issues
Authors:
Jinfu Chen,
Hilary Ackah-Arthur,
Chengying Mao,
Patrick Kwaku Kudjo
Abstract:
Random testing (RT) is a black-box software testing technique that tests programs by generating random test inputs. It is a widely used technique for software quality assurance, but there has been much debate by practitioners concerning its failure-detection effectiveness. RT is argued to be possibly less effective by some researchers as it does not utilize any information about the program under…
▽ More
Random testing (RT) is a black-box software testing technique that tests programs by generating random test inputs. It is a widely used technique for software quality assurance, but there has been much debate by practitioners concerning its failure-detection effectiveness. RT is argued to be possibly less effective by some researchers as it does not utilize any information about the program under test. Efforts to mainly improve the failure-detection capability of RT, have led to the proposition of Adaptive Random Testing (ART). ART takes advantage of the location information of previous non-fault-detecting test cases to enhance effectiveness as compared to RT. The approach has gained popularity and has a large number of theoretical studies and methods that employ different notions. In this review, our goal is to provide an overview of existing ART studies. We classify all ART studies and assess existing ART methods for numeric programs with a focus on their motivation, strategy, and findings. The study also discusses several worthy avenues related to ART. The review uses 109 ART papers in several journals, workshops, and conference proceedings. The results of the review show that significant research efforts have been made towards the field of ART, however further empirical studies are still required to make the technique applicable in different test scenarios in order to impact on the industry.
△ Less
Submitted 28 September, 2019; v1 submitted 24 September, 2019;
originally announced September 2019.
