An Empirical Study on Effects of Code Visibility on Code Coverage of Software Testing

Lei Ma, Cheng Zhang, Bing Yu, Hiroyuki Sato

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Software testability is the degree of difficulty to test a program. Code visibility is important to support design principles, such as information hiding. It is widely believed that code visibility has effects on testability. However, little empirical evidence has been shown to clarify whether and how software testability is influenced by code visibility. We have performed an empirical study to shed light on this problem. Our study focuses on test code coverage, in particular that of automatic testing tools. Code coverage is commonly used for various purposes, such as evaluating test adequacy, assessing test quality, and analyzing testability. Our study uses code coverage as the concrete measurement of testability. By analyzing code coverage of two state-of-the-art tools, in comparison with that of developer-written tests, we have discovered that code visibility does not necessarily have effects on its code coverage, but significantly affects automatic testing tools. Low code visibility often leads to low code coverage for automatic tools. In addition, different treatments on code visibility can result in significant differences in overall code coverage for automatic tools. Using a tool enhancement specific to code visibility, we demonstrate the great potential to improve existing tools.

Original languageEnglish
Title of host publicationProceedings - 10th International Workshop on Automation of Software Test, AST 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages80-84
Number of pages5
ISBN (Electronic)9781467370226
DOIs
Publication statusPublished - Jul 23 2015
Externally publishedYes
Event10th International Workshop on Automation of Software Test, AST 2015 - Florence, Italy
Duration: May 23 2015May 24 2015

Publication series

NameProceedings - 10th International Workshop on Automation of Software Test, AST 2015

Conference

Conference10th International Workshop on Automation of Software Test, AST 2015
CountryItaly
CityFlorence
Period5/23/155/24/15

Fingerprint

Automatic testing
Software testing
Visibility
Concretes

All Science Journal Classification (ASJC) codes

  • Software

Cite this

Ma, L., Zhang, C., Yu, B., & Sato, H. (2015). An Empirical Study on Effects of Code Visibility on Code Coverage of Software Testing. In Proceedings - 10th International Workshop on Automation of Software Test, AST 2015 (pp. 80-84). [7166273] (Proceedings - 10th International Workshop on Automation of Software Test, AST 2015). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/AST.2015.23

An Empirical Study on Effects of Code Visibility on Code Coverage of Software Testing. / Ma, Lei; Zhang, Cheng; Yu, Bing; Sato, Hiroyuki.

Proceedings - 10th International Workshop on Automation of Software Test, AST 2015. Institute of Electrical and Electronics Engineers Inc., 2015. p. 80-84 7166273 (Proceedings - 10th International Workshop on Automation of Software Test, AST 2015).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ma, L, Zhang, C, Yu, B & Sato, H 2015, An Empirical Study on Effects of Code Visibility on Code Coverage of Software Testing. in Proceedings - 10th International Workshop on Automation of Software Test, AST 2015., 7166273, Proceedings - 10th International Workshop on Automation of Software Test, AST 2015, Institute of Electrical and Electronics Engineers Inc., pp. 80-84, 10th International Workshop on Automation of Software Test, AST 2015, Florence, Italy, 5/23/15. https://doi.org/10.1109/AST.2015.23
Ma L, Zhang C, Yu B, Sato H. An Empirical Study on Effects of Code Visibility on Code Coverage of Software Testing. In Proceedings - 10th International Workshop on Automation of Software Test, AST 2015. Institute of Electrical and Electronics Engineers Inc. 2015. p. 80-84. 7166273. (Proceedings - 10th International Workshop on Automation of Software Test, AST 2015). https://doi.org/10.1109/AST.2015.23
Ma, Lei ; Zhang, Cheng ; Yu, Bing ; Sato, Hiroyuki. / An Empirical Study on Effects of Code Visibility on Code Coverage of Software Testing. Proceedings - 10th International Workshop on Automation of Software Test, AST 2015. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 80-84 (Proceedings - 10th International Workshop on Automation of Software Test, AST 2015).
@inproceedings{f5e93687b30249e899f07414f1284615,
title = "An Empirical Study on Effects of Code Visibility on Code Coverage of Software Testing",
abstract = "Software testability is the degree of difficulty to test a program. Code visibility is important to support design principles, such as information hiding. It is widely believed that code visibility has effects on testability. However, little empirical evidence has been shown to clarify whether and how software testability is influenced by code visibility. We have performed an empirical study to shed light on this problem. Our study focuses on test code coverage, in particular that of automatic testing tools. Code coverage is commonly used for various purposes, such as evaluating test adequacy, assessing test quality, and analyzing testability. Our study uses code coverage as the concrete measurement of testability. By analyzing code coverage of two state-of-the-art tools, in comparison with that of developer-written tests, we have discovered that code visibility does not necessarily have effects on its code coverage, but significantly affects automatic testing tools. Low code visibility often leads to low code coverage for automatic tools. In addition, different treatments on code visibility can result in significant differences in overall code coverage for automatic tools. Using a tool enhancement specific to code visibility, we demonstrate the great potential to improve existing tools.",
author = "Lei Ma and Cheng Zhang and Bing Yu and Hiroyuki Sato",
year = "2015",
month = "7",
day = "23",
doi = "10.1109/AST.2015.23",
language = "English",
series = "Proceedings - 10th International Workshop on Automation of Software Test, AST 2015",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "80--84",
booktitle = "Proceedings - 10th International Workshop on Automation of Software Test, AST 2015",
address = "United States",

}

TY - GEN

T1 - An Empirical Study on Effects of Code Visibility on Code Coverage of Software Testing

AU - Ma, Lei

AU - Zhang, Cheng

AU - Yu, Bing

AU - Sato, Hiroyuki

PY - 2015/7/23

Y1 - 2015/7/23

N2 - Software testability is the degree of difficulty to test a program. Code visibility is important to support design principles, such as information hiding. It is widely believed that code visibility has effects on testability. However, little empirical evidence has been shown to clarify whether and how software testability is influenced by code visibility. We have performed an empirical study to shed light on this problem. Our study focuses on test code coverage, in particular that of automatic testing tools. Code coverage is commonly used for various purposes, such as evaluating test adequacy, assessing test quality, and analyzing testability. Our study uses code coverage as the concrete measurement of testability. By analyzing code coverage of two state-of-the-art tools, in comparison with that of developer-written tests, we have discovered that code visibility does not necessarily have effects on its code coverage, but significantly affects automatic testing tools. Low code visibility often leads to low code coverage for automatic tools. In addition, different treatments on code visibility can result in significant differences in overall code coverage for automatic tools. Using a tool enhancement specific to code visibility, we demonstrate the great potential to improve existing tools.

AB - Software testability is the degree of difficulty to test a program. Code visibility is important to support design principles, such as information hiding. It is widely believed that code visibility has effects on testability. However, little empirical evidence has been shown to clarify whether and how software testability is influenced by code visibility. We have performed an empirical study to shed light on this problem. Our study focuses on test code coverage, in particular that of automatic testing tools. Code coverage is commonly used for various purposes, such as evaluating test adequacy, assessing test quality, and analyzing testability. Our study uses code coverage as the concrete measurement of testability. By analyzing code coverage of two state-of-the-art tools, in comparison with that of developer-written tests, we have discovered that code visibility does not necessarily have effects on its code coverage, but significantly affects automatic testing tools. Low code visibility often leads to low code coverage for automatic tools. In addition, different treatments on code visibility can result in significant differences in overall code coverage for automatic tools. Using a tool enhancement specific to code visibility, we demonstrate the great potential to improve existing tools.

UR - http://www.scopus.com/inward/record.url?scp=84945397072&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84945397072&partnerID=8YFLogxK

U2 - 10.1109/AST.2015.23

DO - 10.1109/AST.2015.23

M3 - Conference contribution

AN - SCOPUS:84945397072

T3 - Proceedings - 10th International Workshop on Automation of Software Test, AST 2015

SP - 80

EP - 84

BT - Proceedings - 10th International Workshop on Automation of Software Test, AST 2015

PB - Institute of Electrical and Electronics Engineers Inc.

ER -