An empirical study of the impact of modern code review practices on software quality

Shane McIntosh, Yasutaka Kamei, Bram Adams, Ahmed E. Hassan

Research output: Contribution to journalArticle

59 Citations (Scopus)

Abstract

Software code review, i.e., the practice of having other team members critique changes to a software system, is a well-established best practice in both open source and proprietary software domains. Prior work has shown that formal code inspections tend to improve the quality of delivered software. However, the formal code inspection process mandates strict review criteria (e.g., in-person meetings and reviewer checklists) to ensure a base level of review quality, while the modern, lightweight code reviewing process does not. Although recent work explores the modern code review process, little is known about the relationship between modern code review practices and long-term software quality. Hence, in this paper, we study the relationship between post-release defects (a popular proxy for long-term software quality) and: (1) code review coverage, i.e., the proportion of changes that have been code reviewed, (2) code review participation, i.e., the degree of reviewer involvement in the code review process, and (3) code reviewer expertise, i.e., the level of domain-specific expertise of the code reviewers. Through a case study of the Qt, VTK, and ITK projects, we find that code review coverage, participation, and expertise share a significant link with software quality. Hence, our results empirically confirm the intuition that poorly-reviewed code has a negative impact on software quality in large systems using modern reviewing tools.

Original languageEnglish
Pages (from-to)2146-2189
Number of pages44
JournalEmpirical Software Engineering
Volume21
Issue number5
DOIs
Publication statusPublished - Oct 1 2016

Fingerprint

Inspection
Defects

All Science Journal Classification (ASJC) codes

  • Software

Cite this

An empirical study of the impact of modern code review practices on software quality. / McIntosh, Shane; Kamei, Yasutaka; Adams, Bram; Hassan, Ahmed E.

In: Empirical Software Engineering, Vol. 21, No. 5, 01.10.2016, p. 2146-2189.

Research output: Contribution to journalArticle

McIntosh, Shane ; Kamei, Yasutaka ; Adams, Bram ; Hassan, Ahmed E. / An empirical study of the impact of modern code review practices on software quality. In: Empirical Software Engineering. 2016 ; Vol. 21, No. 5. pp. 2146-2189.
@article{75a5914131794d1ebc577d3db8e48fd9,
title = "An empirical study of the impact of modern code review practices on software quality",
abstract = "Software code review, i.e., the practice of having other team members critique changes to a software system, is a well-established best practice in both open source and proprietary software domains. Prior work has shown that formal code inspections tend to improve the quality of delivered software. However, the formal code inspection process mandates strict review criteria (e.g., in-person meetings and reviewer checklists) to ensure a base level of review quality, while the modern, lightweight code reviewing process does not. Although recent work explores the modern code review process, little is known about the relationship between modern code review practices and long-term software quality. Hence, in this paper, we study the relationship between post-release defects (a popular proxy for long-term software quality) and: (1) code review coverage, i.e., the proportion of changes that have been code reviewed, (2) code review participation, i.e., the degree of reviewer involvement in the code review process, and (3) code reviewer expertise, i.e., the level of domain-specific expertise of the code reviewers. Through a case study of the Qt, VTK, and ITK projects, we find that code review coverage, participation, and expertise share a significant link with software quality. Hence, our results empirically confirm the intuition that poorly-reviewed code has a negative impact on software quality in large systems using modern reviewing tools.",
author = "Shane McIntosh and Yasutaka Kamei and Bram Adams and Hassan, {Ahmed E.}",
year = "2016",
month = "10",
day = "1",
doi = "10.1007/s10664-015-9381-9",
language = "English",
volume = "21",
pages = "2146--2189",
journal = "Empirical Software Engineering",
issn = "1382-3256",
publisher = "Springer Netherlands",
number = "5",

}

TY - JOUR

T1 - An empirical study of the impact of modern code review practices on software quality

AU - McIntosh, Shane

AU - Kamei, Yasutaka

AU - Adams, Bram

AU - Hassan, Ahmed E.

PY - 2016/10/1

Y1 - 2016/10/1

N2 - Software code review, i.e., the practice of having other team members critique changes to a software system, is a well-established best practice in both open source and proprietary software domains. Prior work has shown that formal code inspections tend to improve the quality of delivered software. However, the formal code inspection process mandates strict review criteria (e.g., in-person meetings and reviewer checklists) to ensure a base level of review quality, while the modern, lightweight code reviewing process does not. Although recent work explores the modern code review process, little is known about the relationship between modern code review practices and long-term software quality. Hence, in this paper, we study the relationship between post-release defects (a popular proxy for long-term software quality) and: (1) code review coverage, i.e., the proportion of changes that have been code reviewed, (2) code review participation, i.e., the degree of reviewer involvement in the code review process, and (3) code reviewer expertise, i.e., the level of domain-specific expertise of the code reviewers. Through a case study of the Qt, VTK, and ITK projects, we find that code review coverage, participation, and expertise share a significant link with software quality. Hence, our results empirically confirm the intuition that poorly-reviewed code has a negative impact on software quality in large systems using modern reviewing tools.

AB - Software code review, i.e., the practice of having other team members critique changes to a software system, is a well-established best practice in both open source and proprietary software domains. Prior work has shown that formal code inspections tend to improve the quality of delivered software. However, the formal code inspection process mandates strict review criteria (e.g., in-person meetings and reviewer checklists) to ensure a base level of review quality, while the modern, lightweight code reviewing process does not. Although recent work explores the modern code review process, little is known about the relationship between modern code review practices and long-term software quality. Hence, in this paper, we study the relationship between post-release defects (a popular proxy for long-term software quality) and: (1) code review coverage, i.e., the proportion of changes that have been code reviewed, (2) code review participation, i.e., the degree of reviewer involvement in the code review process, and (3) code reviewer expertise, i.e., the level of domain-specific expertise of the code reviewers. Through a case study of the Qt, VTK, and ITK projects, we find that code review coverage, participation, and expertise share a significant link with software quality. Hence, our results empirically confirm the intuition that poorly-reviewed code has a negative impact on software quality in large systems using modern reviewing tools.

UR - http://www.scopus.com/inward/record.url?scp=84928384936&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84928384936&partnerID=8YFLogxK

U2 - 10.1007/s10664-015-9381-9

DO - 10.1007/s10664-015-9381-9

M3 - Article

AN - SCOPUS:84928384936

VL - 21

SP - 2146

EP - 2189

JO - Empirical Software Engineering

JF - Empirical Software Engineering

SN - 1382-3256

IS - 5

ER -