The impact of using regression models to build defect classifiers

Gopi Krishnan Rajbahadur, Shaowei Wang, Yasutaka Kamei, Ahmed E. Hassan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

12 Citations (Scopus)

Abstract

It is common practice to discretize continuous defect counts into defective and non-defective classes and use them as a target variable when building defect classifiers (discretized classifiers). However, this discretization of continuous defect counts leads to information loss that might affect the performance and interpretation of defect classifiers. Another possible approach to build defect classifiers is through the use of regression models then discretizing the predicted defect counts into defective and non-defective classes (regression-based classifiers). In this paper, we compare the performance and interpretation of defect classifiers that are built using both approaches (i.e., discretized classifiers and regression-based classifiers) across six commonly used machine learning classifiers (i.e., linear/logistic regression, random forest, KNN, SVM, CART, and neural networks) and 17 datasets. We find that: i) Random forest based classifiers outperform other classifiers (best AUC) for both classifier building approaches, ii) In contrast to common practice, building a defect classifier using discretized defect counts (i.e., discretized classifiers) does not always lead to better performance. Hence we suggest that future defect classification studies should consider building regression-based classifiers (in particular when the defective ratio of the modeled dataset is low). Moreover, we suggest that both approaches for building defect classifiers should be explored, so the best-performing classifier can be used when determining the most influential features.

Original languageEnglish
Title of host publicationProceedings - 2017 IEEE/ACM 14th International Conference on Mining Software Repositories, MSR 2017
PublisherIEEE Computer Society
Pages135-145
Number of pages11
ISBN (Electronic)9781538615447
DOIs
Publication statusPublished - Jun 29 2017
Event14th IEEE/ACM International Conference on Mining Software Repositories, MSR 2017 - Buenos Aires, Argentina
Duration: May 20 2017May 21 2017

Publication series

NameIEEE International Working Conference on Mining Software Repositories
ISSN (Print)2160-1852
ISSN (Electronic)2160-1860

Other

Other14th IEEE/ACM International Conference on Mining Software Repositories, MSR 2017
CountryArgentina
CityBuenos Aires
Period5/20/175/21/17

Fingerprint

Classifiers
Defects
Learning systems
Logistics

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Software

Cite this

Rajbahadur, G. K., Wang, S., Kamei, Y., & Hassan, A. E. (2017). The impact of using regression models to build defect classifiers. In Proceedings - 2017 IEEE/ACM 14th International Conference on Mining Software Repositories, MSR 2017 (pp. 135-145). [7962363] (IEEE International Working Conference on Mining Software Repositories). IEEE Computer Society. https://doi.org/10.1109/MSR.2017.4

The impact of using regression models to build defect classifiers. / Rajbahadur, Gopi Krishnan; Wang, Shaowei; Kamei, Yasutaka; Hassan, Ahmed E.

Proceedings - 2017 IEEE/ACM 14th International Conference on Mining Software Repositories, MSR 2017. IEEE Computer Society, 2017. p. 135-145 7962363 (IEEE International Working Conference on Mining Software Repositories).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Rajbahadur, GK, Wang, S, Kamei, Y & Hassan, AE 2017, The impact of using regression models to build defect classifiers. in Proceedings - 2017 IEEE/ACM 14th International Conference on Mining Software Repositories, MSR 2017., 7962363, IEEE International Working Conference on Mining Software Repositories, IEEE Computer Society, pp. 135-145, 14th IEEE/ACM International Conference on Mining Software Repositories, MSR 2017, Buenos Aires, Argentina, 5/20/17. https://doi.org/10.1109/MSR.2017.4
Rajbahadur GK, Wang S, Kamei Y, Hassan AE. The impact of using regression models to build defect classifiers. In Proceedings - 2017 IEEE/ACM 14th International Conference on Mining Software Repositories, MSR 2017. IEEE Computer Society. 2017. p. 135-145. 7962363. (IEEE International Working Conference on Mining Software Repositories). https://doi.org/10.1109/MSR.2017.4
Rajbahadur, Gopi Krishnan ; Wang, Shaowei ; Kamei, Yasutaka ; Hassan, Ahmed E. / The impact of using regression models to build defect classifiers. Proceedings - 2017 IEEE/ACM 14th International Conference on Mining Software Repositories, MSR 2017. IEEE Computer Society, 2017. pp. 135-145 (IEEE International Working Conference on Mining Software Repositories).
@inproceedings{414974cc4245449cab7fdf6c49688c0c,
title = "The impact of using regression models to build defect classifiers",
abstract = "It is common practice to discretize continuous defect counts into defective and non-defective classes and use them as a target variable when building defect classifiers (discretized classifiers). However, this discretization of continuous defect counts leads to information loss that might affect the performance and interpretation of defect classifiers. Another possible approach to build defect classifiers is through the use of regression models then discretizing the predicted defect counts into defective and non-defective classes (regression-based classifiers). In this paper, we compare the performance and interpretation of defect classifiers that are built using both approaches (i.e., discretized classifiers and regression-based classifiers) across six commonly used machine learning classifiers (i.e., linear/logistic regression, random forest, KNN, SVM, CART, and neural networks) and 17 datasets. We find that: i) Random forest based classifiers outperform other classifiers (best AUC) for both classifier building approaches, ii) In contrast to common practice, building a defect classifier using discretized defect counts (i.e., discretized classifiers) does not always lead to better performance. Hence we suggest that future defect classification studies should consider building regression-based classifiers (in particular when the defective ratio of the modeled dataset is low). Moreover, we suggest that both approaches for building defect classifiers should be explored, so the best-performing classifier can be used when determining the most influential features.",
author = "Rajbahadur, {Gopi Krishnan} and Shaowei Wang and Yasutaka Kamei and Hassan, {Ahmed E.}",
year = "2017",
month = "6",
day = "29",
doi = "10.1109/MSR.2017.4",
language = "English",
series = "IEEE International Working Conference on Mining Software Repositories",
publisher = "IEEE Computer Society",
pages = "135--145",
booktitle = "Proceedings - 2017 IEEE/ACM 14th International Conference on Mining Software Repositories, MSR 2017",
address = "United States",

}

TY - GEN

T1 - The impact of using regression models to build defect classifiers

AU - Rajbahadur, Gopi Krishnan

AU - Wang, Shaowei

AU - Kamei, Yasutaka

AU - Hassan, Ahmed E.

PY - 2017/6/29

Y1 - 2017/6/29

N2 - It is common practice to discretize continuous defect counts into defective and non-defective classes and use them as a target variable when building defect classifiers (discretized classifiers). However, this discretization of continuous defect counts leads to information loss that might affect the performance and interpretation of defect classifiers. Another possible approach to build defect classifiers is through the use of regression models then discretizing the predicted defect counts into defective and non-defective classes (regression-based classifiers). In this paper, we compare the performance and interpretation of defect classifiers that are built using both approaches (i.e., discretized classifiers and regression-based classifiers) across six commonly used machine learning classifiers (i.e., linear/logistic regression, random forest, KNN, SVM, CART, and neural networks) and 17 datasets. We find that: i) Random forest based classifiers outperform other classifiers (best AUC) for both classifier building approaches, ii) In contrast to common practice, building a defect classifier using discretized defect counts (i.e., discretized classifiers) does not always lead to better performance. Hence we suggest that future defect classification studies should consider building regression-based classifiers (in particular when the defective ratio of the modeled dataset is low). Moreover, we suggest that both approaches for building defect classifiers should be explored, so the best-performing classifier can be used when determining the most influential features.

AB - It is common practice to discretize continuous defect counts into defective and non-defective classes and use them as a target variable when building defect classifiers (discretized classifiers). However, this discretization of continuous defect counts leads to information loss that might affect the performance and interpretation of defect classifiers. Another possible approach to build defect classifiers is through the use of regression models then discretizing the predicted defect counts into defective and non-defective classes (regression-based classifiers). In this paper, we compare the performance and interpretation of defect classifiers that are built using both approaches (i.e., discretized classifiers and regression-based classifiers) across six commonly used machine learning classifiers (i.e., linear/logistic regression, random forest, KNN, SVM, CART, and neural networks) and 17 datasets. We find that: i) Random forest based classifiers outperform other classifiers (best AUC) for both classifier building approaches, ii) In contrast to common practice, building a defect classifier using discretized defect counts (i.e., discretized classifiers) does not always lead to better performance. Hence we suggest that future defect classification studies should consider building regression-based classifiers (in particular when the defective ratio of the modeled dataset is low). Moreover, we suggest that both approaches for building defect classifiers should be explored, so the best-performing classifier can be used when determining the most influential features.

UR - http://www.scopus.com/inward/record.url?scp=85026518861&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85026518861&partnerID=8YFLogxK

U2 - 10.1109/MSR.2017.4

DO - 10.1109/MSR.2017.4

M3 - Conference contribution

AN - SCOPUS:85026518861

T3 - IEEE International Working Conference on Mining Software Repositories

SP - 135

EP - 145

BT - Proceedings - 2017 IEEE/ACM 14th International Conference on Mining Software Repositories, MSR 2017

PB - IEEE Computer Society

ER -