University of Minnesota
Software Engineering Center

You are here

Automated Oracle Creation Support, or: How I Learned to Stop Worrying About Fault Propagation and Love Mutation Testing

Date of Publication: 
May 2012
Associated Research Groups: 
Publication Files: 
In testing, the test oracle is the artifact that determines whether an application under test executes correctly. The choice of test oracle can significantly impact the effectiveness of the testing process. However, despite the prevalence of tools that support the selection of test inputs, little work exists for supporting oracle creation. In this work, we propose a method of supporting test oracle creation. This method automatically selects the oracle data - the set of variables monitored during testing - for expected value test oracles. This approach is based on the use of mutation analysis to rank variables in terms of fault-finding effectiveness, thus automating the selection of the oracle data. Experiments over four industrial examples demonstrate that our method may be a cost-effective approach for producing small, effective oracle data, with fault finding improvements over current industrial best practice of up to 145.8% observed.
IEEE Press
Proceedings of the 2012 International Conference on Software Engineering
@inproceedings{Staats:2012:AOC:2337223.2337326, author = {Staats, Matt and Gay, Gregory and Heimdahl, Mats P. E.}, title = {Automated oracle creation support, or: how I learned to stop worrying about fault propagation and love mutation testing}, booktitle = {Proceedings of the 2012 International Conference on Software Engineering}, series = {ICSE 2012}, year = {2012}, isbn = {978-1-4673-1067-3}, location = {Zurich, Switzerland}, pages = {870--880}, numpages = {11}, url = {}, acmid = {2337326}, publisher = {IEEE Press}, address = {Piscataway, NJ, USA}, }