Faster Creation of Smaller Test Suites (with SNAP)
share
State-of-the-art theorem provers, combined with smart sampling heuristics, can generate millions of test cases in just a few hours. But given the heuristic nature of those methods, not all of those tests may be valid. Also, test engineers may find it too burdensome to run all those tests. Within a large space of tests, there can be redundancies (duplicate entries or similar entries that do not contribute much to overall diversity). Our approach, called SNAP uses specialized sub-sampling heuristics to avoid finding those repeated tests. By avoiding those repeated structures SNAP explores a smaller space of options. Hence, it is possible for SNAP to verify all its tests. To evaluate SNAP, this paper applied 27 real-world case studies from a recent ICSE'18 paper. Compared to prior results, SNAP's test case generation was 10 to 3000 times faster (median to max). Also, while prior work showed that their tests were 70 test engineers would find it relatively easiest to apply SNAP's tests since our test suites are 10 to 750 times smaller (median to max) than those generated using prior work.
READ FULL TEXT