Abstract test case prioritization using repeated small-strength level-combination coverage

Huang, Rubing and Sun, Weifeng and Chen, Tsong Yueh and Towey, Dave and Chen, Jinfu and Zong, Weiwen and Zhou, Yunan (2020) Abstract test case prioritization using repeated small-strength level-combination coverage. IEEE Transactions on Reliability, 69 (1). pp. 349-372. ISSN 0018-9529

[img]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Available under Licence Creative Commons Attribution.
Download (24MB) | Preview

Abstract

Abstract—Abstract Test Cases (ATCs) have been widely used in practice, including in combinatorial testing and in software product line testing. When constructing a set of ATCs, due to limited testing resources in practice (for example in regression testing), Test Case Prioritization (TCP) has been proposed to improve the testing quality, aiming at ordering test cases to increase the speed with which faults are detected. One intuitive and extensively studied TCP technique for ATCs is λ-wise Level-combination Coverage based Prioritization (λLCP), a static, black-box prioritization technique that only uses the ATC information to guide the prioritization process. A challenge facing λLCP, however, is the necessity for the selection of the fixed prioritization strength λ before testing — testers need to choose an appropriate λ value before testing begins. Choosing higher λ values may improve the testing effectiveness of λLCP (for example, by finding faults faster), but may reduce the testing efficiency (by incurring additional prioritization costs). Conversely, choosing lower λ values may improve the efficiency, but may also reduce the effectiveness. In this paper, we propose a new family of λLCP techniques, Repeated Small-strength Level-combination Coverage-based Prioritization (RSLCP), that repeatedly achieves the full combination coverage at lower strengths. RSLCP maintains λLCP’s advantages of being static and black box, but avoids the challenge of prioritization strength selection. We performed an empirical study involving five different versions of each of five C programs. Compared with λLCP, and Incremental strength LCP (ILCP), our results show that RSLCP could provide a good trade-off between testing effectiveness and efficiency. Our results also show that RSLCP is more effective and efficient than two popular techniques of Similarity-based Prioritization (SP). In addition, the results of empirical studies also show that RSLCP can remain robust over multiple system releases.

Item Type: Article
Keywords: Software testing, regression testing, abstract test case, test case prioritization, level-combination coverage
Schools/Departments: University of Nottingham Ningbo China > Faculty of Science and Engineering > School of Computer Science
Identification Number: https://doi.org/10.1109/TR.2019.2908068
Depositing User: Zhou, Elsie
Date Deposited: 13 Apr 2020 01:38
Last Modified: 13 Apr 2020 01:38
URI: http://eprints.nottingham.ac.uk/id/eprint/60295

Actions (Archive Staff Only)

Edit View Edit View