University of Leicester
Browse

Random or Evolutionary Search for Object-Oriented Test Suite Generation?

Download (298.23 kB)
journal contribution
posted on 2019-05-22, 09:21 authored by S Shamshiri, JM Rojas, L Gazzola, G Fraser, P McMinn, L Mariani, A Arcuri
An important aim in software testing is constructing a test suite with high structural code coverage, that is, ensuring that most if not all of the code under test have been executed by the test cases comprising the test suite. Several search‐based techniques have proved successful at automatically generating tests that achieve high coverage. However, despite the well‐established arguments behind using evolutionary search algorithms (eg, genetic algorithms) in preference to random search, it remains an open question whether the benefits can actually be observed in practice when generating unit test suites for object‐oriented classes. In this paper, we report an empirical study on the effects of using evolutionary algorithms (including a genetic algorithm and chemical reaction optimization) to generate test suites, compared with generating test suites incrementally with random search. We apply the EVOSUITE unit test suite generator to 1000 classes randomly selected from the SF110 corpus of open‐source projects. Surprisingly, the results show that the difference is much smaller than one might expect: While evolutionary search covers more branches of the type where standard fitness functions provide guidance, we observed that, in practice, the vast majority of branches do not provide any guidance to the search. These results suggest that, although evolutionary algorithms are more effective at covering complex branches, a random search may suffice to achieve high coverage of most object‐oriented classes.

Funding

This work is supported by the EPSRC project “GREATEST” (EP/N023978/1), by the National Research Fund, Luxembourg (FNR/P10/03), and by the H2020 Learn project, which has been funded under the ERC Consolidator Grant 2014 program (ERC Grant Agreement no. 646867).

History

Citation

Software Testing, Verification and Reliability (STVR), 2018, 28(4), e1660

Author affiliation

/Organisation/COLLEGE OF SCIENCE AND ENGINEERING/Department of Informatics

Version

  • AM (Accepted Manuscript)

Published in

Software Testing

Publisher

Wiley

eissn

1099-1689

Acceptance date

2017-12-14

Copyright date

2018

Available date

2019-05-22

Publisher version

https://onlinelibrary.wiley.com/doi/full/10.1002/stvr.1660

Language

en

Usage metrics

    University of Leicester Publications

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC