The impact of code review coverage and code review participation on Software quality: A case study of the Qt, VTK, and ITK projects

Shane McIntosh, Yasutaka Kamei, Bram Adams, Ahmed E. Hassan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

121 Citations (Scopus)

Abstract

Software code review, i.e., the practice of having third-party team members critique changes to a software system, is a well-established best practice in both open source and proprietary software domains. Prior work has shown that the formal code inspections of the past tend to improve the quality of software delivered by students and small teams. However, the formal code inspection process mandates strict review criteria (e.g., in-person meetings and reviewer checklists) to ensure a base level of review quality, while the modern, lightweight code reviewing process does not. Although recent work explores the modern code review process qualitatively, little research quantitatively explores the relationship between properties of the modern code review process and software quality. Hence, in this paper, we study the relationship between software quality and: (1) code review coverage, i.e., the proportion of changes that have been code reviewed, and (2) code review participation, i.e., the degree of reviewer involvement in the code review process. Through a case study of the Qt, VTK, and ITK projects, we find that both code review coverage and participation share a significant link with software quality. Low code review coverage and participation are estimated to produce components with up to two and five additional post-release defects respectively. Our results empirically confirm the intuition that poorly reviewed code has a negative impact on software quality in large systems using modern reviewing tools.

Original languageEnglish
Title of host publication11th Working Conference on Mining Software Repositories, MSR 2014 - Proceedings
PublisherAssociation for Computing Machinery, Inc
Pages192-201
Number of pages10
ISBN (Electronic)9781450328630
DOIs
Publication statusPublished - May 31 2014
Event11th International Working Conference on Mining Software Repositories, MSR 2014 - Hyderabad, India
Duration: May 31 2014Jun 1 2014

Publication series

Name11th Working Conference on Mining Software Repositories, MSR 2014 - Proceedings

Other

Other11th International Working Conference on Mining Software Repositories, MSR 2014
CountryIndia
CityHyderabad
Period5/31/146/1/14

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Software

Fingerprint Dive into the research topics of 'The impact of code review coverage and code review participation on Software quality: A case study of the Qt, VTK, and ITK projects'. Together they form a unique fingerprint.

  • Cite this

    McIntosh, S., Kamei, Y., Adams, B., & Hassan, A. E. (2014). The impact of code review coverage and code review participation on Software quality: A case study of the Qt, VTK, and ITK projects. In 11th Working Conference on Mining Software Repositories, MSR 2014 - Proceedings (pp. 192-201). (11th Working Conference on Mining Software Repositories, MSR 2014 - Proceedings). Association for Computing Machinery, Inc. https://doi.org/10.1145/2597073.2597076