Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userA Survey on Adaptive Random Testing
137
0
0.0
(
0
)
Added by
Weifeng Sun
Publication date
2020
fields
Informatics Engineering
and research's language is
English
Ask ChatGPT about the research
No Arabic abstract
Random testing (RT) is a well-studied testing method that has been widely applied to the testing of many applications, including embedded software systems, SQL database systems, and Android applications. Adaptive random testing (ART) aims to enhance RTs failure-detection ability by more evenly spreading the test cases over the input domain. Since its introduction in 2001, there have been many contributions to the development of ART, including various approaches, implementations, assessment and evaluation methods, and applications. This paper provides a comprehensive survey on ART, classifying techniques, summarizing application areas, and analyzing experimental evaluations. This paper also addresses some misconceptions about ART, and identifies open research challenges to be further investigated in the future work.
rate research
Read More
Ensuring correctness of cyber-physical systems (CPS) is an extremely challenging task that is in practice often addressed with simulation based testing. Formal specification languages, such as Signal Temporal Logic (STL), are used to mathematically express CPS requirements and thus render the simulation activity more systematic and principled. We propose a novel method for adaptive generation of tests with specification coverage for STL. To achieve this goal, we devise cooperative reachability games that we combine with numerical optimization to create tests that explore the system in a way that exercise various parts of the specification. To the best of our knowledge our approach is the first adaptive testing approach that can be applied directly to MATLABtexttrademark; Simulink/Stateflow models. We implemented our approach in a prototype tool and evaluated it on several illustrating examples and a case study from the avionics domain, demonstrating the effectiveness of adaptive testing to (1) incrementally build a test case that reaches a test objective, (2) generate a test suite that increases the specification coverage, and (3) infer what part of the specification is actually implemented.
Adaptive random testing (ART) improves the failure-detection effectiveness of random testing by leveraging properties of the clustering of failure-causing inputs of most faulty programs: ART uses a sampling mechanism that evenly spreads test cases within a softwares input domain. The widely-used Fixed-Sized-Candidate-Set ART (FSCS-ART) sampling strategy faces a quadratic time cost, which worsens as the dimensionality of the software input domain increases. In this paper, we propose an approach based on small world graphs that can enhance the computational efficiency of FSCS-ART: SWFC-ART. To efficiently perform nearest neighbor queries for candidate test cases, SWFC-ART incrementally constructs a hierarchical navigable small world graph for previously executed, non-failure-causing test cases. Moreover, SWFC-ART has shown consistency in programs with high dimensional input domains. Our simulation and empirical studies show that SWFC-ART reduces the computational overhead of FSCS-ART from quadratic to log-linear order while maintaining the failure-detection effectiveness of FSCS-ART, and remaining consistent in high dimensional input domains. We recommend using SWFC-ART in practical software testing scenarios, where real-life programs often have high dimensional input domains and low failure rates.
As an enhanced version of Random Testing (RT), Adaptive Random Testing (ART) aims to improve the failure detection effectiveness of RT by distributing the test cases more evenly in the input domain. Many ART algorithms have been proposed based on different criteria. Among them, the Fixed-Size-Candidate-Set ART (FSCS-ART) is one of the most effective and classical algorithms. FSCS-ART ensures high failure detection effectiveness using selecting the candidate test case which farthest from the previously executed test cases as the next test case. Although FSCS-ART has good failure detection effectiveness, it also has some drawbacks, such as computational overhead problem. In this paper, we propose an enhanced version of FSCS-ART, namely Vantage Point Partitioning Strategy based ART (VPPS-ART). VPPS-ART attempts to solve the computational overhead problem of FSCS-ART using vantage point partitioning strategy and ensures the failure detection effectiveness of FSCS-ART. VPPS-ART achieves the partitioning of the input domain space by using a Vantage Point tree (VP-tree) and finds the nearest executed test cases of a candidate test case in the partitioned sub-domains, which reduces the time overhead significantly compared to the entire input domain search computation. Besides, to match the dynamic insertion process of FSCS-ART, we modify the structure of the traditional VP-tree to support dynamic data. The simulation results present that VPPS-ART has a great lower time overhead compared to FSCS-ART, and also guarantees similar or better failure detection effectiveness than FSCS-ART. The VPPS-ART also shows strength in comparison with the KDFC-ART algorithms, a series of enhanced ART algorithms based on KD-tree. Our empirical studies also reveal that VPPS-ART is more cost-effective compared to FSCS-ART and KDFC-ART.
The success of several constraint-based modeling languages such as OPL, ZINC, or COMET, appeals for better software engineering practices, particularly in the testing phase. This paper introduces a testing framework enabling automated test case generation for constraint programming. We propose a general framework of constraint program development which supposes that a first declarative and simple constraint model is available from the problem specifications analysis. Then, this model is refined using classical techniques such as constraint reformulation, surrogate and global constraint addition, or symmetry-breaking to form an improved constraint model that must be thoroughly tested before being used to address real-sized problems. We think that most of the faults are introduced in this refinement step and propose a process which takes the first declarative model as an oracle for detecting non-conformities. We derive practical test purposes from this process to generate automatically test data that exhibit non-conformities. We implemented this approach in a new tool called CPTEST that was used to automatically detect non-conformities on two classical benchmark programs, namely the Golomb rulers and the car-sequencing problem.
This note concerns a search for publications in which the pragmatic concept of a test as conducted in the practice of software testing is formalized, a theory about software testing based on such a formalization is presented or it is demonstrated on the basis of such a theory that there are solid grounds to test software in cases where in principle other forms of analysis could be used. This note reports on the way in which the search has been carried out and the main outcomes of the search. The message of the note is that the fundamentals of software testing are not yet complete in some respects.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
