Empirical Evaluation of Cross-Release Effort-Aware Defect Prediction Models

Kwabena Ebo Bennin, Koji Toda, Yasutaka Kamei, Jacky Keung, Akito Monden, Naoyasu Ubayashi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

18 Citations (Scopus)

Abstract

To prioritize quality assurance efforts, various fault prediction models have been proposed. However, the best performing fault prediction model is unknown due to three major drawbacks: (1) comparison of few fault prediction models considering small number of data sets, (2) use of evaluation measures that ignore testing efforts and (3) use of n-fold cross-validation instead of the more practical cross-release validation. To address these concerns, we conducted cross-release evaluation of 11 fault density prediction models using data sets collected from 2 releases of 25 open source software projects with an effort-Aware performance measure known as Norm(Popt). Our result shows that, whilst M5 and K∗ had the best performances, they were greatly influenced by the percentage of faulty modules present and size of data set. Using Norm(Popt) produced an overall average performance of more than 50% across all the selected models clearly indicating the importance of considering testing efforts in building fault-prone prediction models.

Original languageEnglish
Title of host publicationProceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security, QRS 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages214-221
Number of pages8
ISBN (Electronic)9781509041275
DOIs
Publication statusPublished - Oct 12 2016
Externally publishedYes
Event2nd IEEE International Conference on Software Quality, Reliability and Security, QRS 2016 - Vienna, Austria
Duration: Aug 1 2016Aug 3 2016

Publication series

NameProceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security, QRS 2016

Other

Other2nd IEEE International Conference on Software Quality, Reliability and Security, QRS 2016
CountryAustria
CityVienna
Period8/1/168/3/16

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Software
  • Safety, Risk, Reliability and Quality

Fingerprint Dive into the research topics of 'Empirical Evaluation of Cross-Release Effort-Aware Defect Prediction Models'. Together they form a unique fingerprint.

  • Cite this

    Bennin, K. E., Toda, K., Kamei, Y., Keung, J., Monden, A., & Ubayashi, N. (2016). Empirical Evaluation of Cross-Release Effort-Aware Defect Prediction Models. In Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security, QRS 2016 (pp. 214-221). [7589801] (Proceedings - 2016 IEEE International Conference on Software Quality, Reliability and Security, QRS 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/QRS.2016.33