The effects of change decomposition on code review - a controlled experiment

, , ,
1 Delft University of Technology, Delft, The Netherlands
2 Software Improvement Group, Amsterdam, The Netherlands
3 University of Zurich, Zurich, Switzerland
DOI
10.7287/peerj.preprints.27438v1
Published
Accepted
Subject Areas
Human-Computer Interaction, Software Engineering
Keywords
code review, controlled experiment, change decomposition, pull-based development model
Copyright
© 2018 di Biase et al.
Licence
This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Preprints) and either DOI or URL of the article must be cited.
Cite this article
di Biase M, Bruntink M, van Deursen A, Bacchelli A. 2018. The effects of change decomposition on code review - a controlled experiment. PeerJ Preprints 6:e27438v1

Abstract

Background. Code review is a cognitively demanding and time-consuming process. Previous qualitative studies hinted at how decomposing change sets into multiple yet internally coherent ones would improve the reviewing process. So far, no quantitative analysis of this hypothesis has been provided.

Aims. (1) Quantitatively measure the effects of change decomposition on the outcome of code review (in terms of number of found defects, wrongly reported issues, suggested improvements, time, and understanding); (2) Qualitatively analyze how subjects approach the review and navigate the code, building knowledge and addressing existing issues, in large vs. decomposed changes.

Method. Controlled experiment using the pull-based development model involving 28 software developers among professionals and graduate students.

Results. Change decomposition leads to fewer wrongly reported issues, influences how subjects approach and conduct the review activity (by increasing context-seeking), yet impacts neither understanding the change rationale nor the number of found defects.

Conclusions. Change decomposition reduces the noise for subsequent data analyses but also significantly supports the tasks of the developers in charge of reviewing the changes. As such, commits belonging to different concepts should be separated, adopting this as a best practice in software engineering.

Author Comment

This is a submission to PeerJ Computer Science for review.