The unreasonable effectiveness of traditional information retrieval in crash report deduplication
- Published
- Accepted
- Subject Areas
- Data Mining and Machine Learning, Software Engineering
- Keywords
- crash report deduplication, automated crash reporting, software engineering, information retrieval, tf-idf
- Copyright
- © 2016 Campbell et al.
- Licence
- This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ PrePrints) and either DOI or URL of the article must be cited.
- Cite this article
- 2016. The unreasonable effectiveness of traditional information retrieval in crash report deduplication. PeerJ PrePrints 4:e1705v1 https://doi.org/10.7287/peerj.preprints.1705v1
Abstract
Organizations like Mozilla, Microsoft, and Apple are flooded with thousands of automated crash reports per day. Although crash reports contain valuable information for debugging, there are often too many for developers to examine individually. Therefore, in industry, crash reports are often automatically grouped together in buckets. Ubuntu’s repository contains crashes from hundreds of software systems available with Ubuntu. A variety of crash report bucketing methods are evaluated using data collected by Ubuntu’s Apport automated crash reporting system. The trade-off between precision and recall of numerous scalable crash 7 deduplication techniques is explored. A set of criteria that a crash deduplication method must meet is presented and several methods that meet these criteria are evaluated on a new dataset. The evaluations presented in this paper show that using off-the-shelf information retrieval techniques, that were not designed to be used with crash reports, outperform other techniques which are specifically designed for the task of crash bucketing at realistic industrial scales. This research indicates that automated crash bucketing still has a lot of room for improvement, especially in terms of identifier tokenization.
Author Comment
This is a preprint submission to PeerJ Preprints. Submitted to MSR 2016 for peer review.