MTGP: Combining Metamorphic Testing and Genetic Programming

by   Dominik Sobania, et al.

Genetic programming is an evolutionary approach known for its performance in program synthesis. However, it is not yet mature enough for a practical use in real-world software development, since usually many training cases are required to generate programs that generalize to unseen test cases. As in practice, the training cases have to be expensively hand-labeled by the user, we need an approach to check the program behavior with a lower number of training cases. Metamorphic testing needs no labeled input/output examples. Instead, the program is executed multiple times, first on a given (randomly generated) input, followed by related inputs to check whether certain user-defined relations between the observed outputs hold. In this work, we suggest MTGP, which combines metamorphic testing and genetic programming and study its performance and the generalizability of the generated programs. Further, we analyze how the generalizability depends on the number of given labeled training cases. We find that using metamorphic testing combined with labeled training cases leads to a higher generalization rate than the use of labeled training cases alone in almost all studied configurations. Consequently, we recommend researchers to use metamorphic testing in their systems if the labeling of the training data is expensive.


page 1

page 2

page 3

page 4


Choose Your Programming Copilot: A Comparison of the Program Synthesis Performance of GitHub Copilot and Genetic Programming

GitHub Copilot, an extension for the Visual Studio Code development envi...

The Composability of Intermediate Values in Composable Inductive Programming

It is believed that mechanisms including intermediate values enable comp...

Toward Measuring the Scaling of Genetic Programming

Several genetic programming systems are created, each solving a differen...

Extracting Formal Specifications to Strenghten Type Behaviour Testing

Testing has become an indispensable activity of software development, ye...

Data Generation for Neural Programming by Example

Programming by example is the problem of synthesizing a program from a s...

Enhancing POI testing approach through the use of additional information

Recently, a new approach to perform regression testing has been defined:...

FuzzyFlow: Leveraging Dataflow To Find and Squash Program Optimization Bugs

The current hardware landscape and application scale is driving performa...

Please sign up or login with your details

Forgot password? Click here to reset