Making computer science results reproducible - A case study using Gradle and Docker
- Published
- Accepted
- Subject Areas
- Data Science, Scientific Computing and Simulation, Software Engineering
- Keywords
- in-silico research, reproducibility, simulation
- Copyright
- © 2018 Elmenreich et al.
- Licence
- This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Preprints) and either DOI or URL of the article must be cited.
- Cite this article
- 2018. Making computer science results reproducible - A case study using Gradle and Docker. PeerJ Preprints 6:e27082v1 https://doi.org/10.7287/peerj.preprints.27082v1
Abstract
This paper addresses two questions related to reproducibility within the context of research related to computer science. First, requirements on reproducibility are analyzed based on a survey addressed to researchers in the academic and private sector. The survey indicates a strong need for open but also easily accessible results, thus reproducing an experiment should not require too much effort. The results from the survey are then used to formulate general guidelines for making research results reproducible. In addition, this paper explores a number of existing software tools that could bring forward reproducibility in research results. After a general analysis of tools a further investigation is done via three case studies based on actual research projects which are used to evaluate the previously introduced tools. Results indicate that due to conflicting requirements, none of the presented solutions fulfills all intended goals perfectly. However, we present requirements and guidelines for making research reproducible. While the main focus of this paper is on reproducibility in computer science, the results of this paper are still valid for other fields using computation as a tool.
Author Comment
This is a submission to PeerJ Computer Science for review.
Supplemental Information
Responses from online survey on reproducibility in software-based research and engineering
Responses from online survey on reproducibility in software-based research and engineering.