Double-blind review in software engineering venues
Author and article information
Abstract
The peer review process is central to the scientific method, the advancement and spread of research, as well as crucial for individual careers. However, the single-blind review mode currently used in most Software Engineering (SE) venues is susceptible to apparent and hidden biases, since reviewers know the identity of authors. We perform a study on the benefits and costs that are associated with introducing double- blind review in SE venues. We surveyed the SE community’s opinion and interviewed experts on double-blind reviewing. Our results indicate that the costs, mostly logistic challenges and side effects, outnumber its benefits and mostly regard difficulty for authors in blinding papers, for reviewers in understanding the increment with respect to previous work from the same authors, and for organizers to manage a complex transition. While the surveyed community largely consents on the costs of DBR, only less than one-third disagree with a switch to DBR for SE journals, all SE conferences, and, in particular, ICSE; the analysis of a survey with authors of submitted papers at ICSE 2016 run by the program chairs of that edition corroborates our result.
Cite this as
2016. Double-blind review in software engineering venues. PeerJ Preprints 4:e1757v2 https://doi.org/10.7287/peerj.preprints.1757v2Author comment
In this version, we added an analysis based on a post-ICSE'16 survey and refined our discussion of threats to the validity of our study.
Sections
Additional Information
Competing Interests
The authors declare that they have no competing interests.
Author Contributions
Moritz Beller conceived and designed the experiments, performed the experiments, analyzed the data, contributed reagents/materials/analysis tools, wrote the paper, prepared figures and/or tables, performed the computation work, reviewed drafts of the paper.
Alberto Bacchelli conceived and designed the experiments, performed the experiments, analyzed the data, contributed reagents/materials/analysis tools, wrote the paper, prepared figures and/or tables, performed the computation work, reviewed drafts of the paper.
Data Deposition
The following information was supplied regarding data availability:
Data collection not yet finished. Will be update.
Funding
The authors received no funding for this work.
