How systematic reviews cover practitioners' issues: A study on Stack Exchange communities
- Published
- Accepted
- Subject Areas
- Software Engineering
- Keywords
- Systematic Reviews, Stack Exchange, Evidence-Based Software Engineering, Software Engineering Practice
- Copyright
- © 2019 Cartaxo et al.
- Licence
- This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Preprints) and either DOI or URL of the article must be cited.
- Cite this article
- 2019. How systematic reviews cover practitioners' issues: A study on Stack Exchange communities. PeerJ Preprints 7:e27610v1 https://doi.org/10.7287/peerj.preprints.27610v1
Abstract
Context: One of the goals of Evidence-Based Software Engineering is to leverage evidence from research to practice. However, some studies suggest this goal has not being fully accomplished.
Objective: This paper proposes a strategy to assess how systematic reviews cover practitioners' issues in software engineering.
Method: We selected 24 systematic reviews identified by a comprehensive tertiary study. Using search strings of the selected systematic reviews, we queried most relevant practitioners' issues on five active Stack Exchange communities, a professional and high-quality Question & Answer platform. After examining more than 1,800 issues, we investigated how findings of the selected systematic reviews could help to solve (i.e. cover) practitioners' issues.
Results: After excluding false positives and duplicates, a total of 424 issues were considered related to the selected systematic reviews. This number corresponds to 1.75% of the 26,687 most relevant issues on the five Stack Exchange communities. Among these 424 issues, systematic reviews can successfully cover 14.1% (60) of them. Based on a qualitative analysis, we identified 45 recurrent issues spread in many software engineering areas. The most demanded topic is related to agile software development, with 15 recurrent issues identified and 127 practitioners' issues as a whole.
Conclusions: An overall coverage rate of 14.1% reveals a good opportunity for conducting systematic reviews in software engineering to fill the gap of not covered issues. We also observed practitioners explicitly demanding for scientific empirical evidence, rich in context and oriented to specific target audiences. Finally, we also provided guidelines for researchers who want to conduct systematic reviews more connected with software engineering practice.
Author Comment
This is a submission to PeerJ Computer Science for review.
