Accelerating the XGBoost algorithm using GPU computing
- Published
- Accepted
- Subject Areas
- Artificial Intelligence, Data Mining and Machine Learning
- Keywords
- supervised machine learning, gradient boosting, GPU computing
- Copyright
- © 2017 Mitchell et al.
- Licence
- This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Preprints) and either DOI or URL of the article must be cited.
- Cite this article
- 2017. Accelerating the XGBoost algorithm using GPU computing. PeerJ Preprints 5:e2911v1 https://doi.org/10.7287/peerj.preprints.2911v1
Abstract
We present a CUDA based implementation of a decision tree construction algorithm within the gradient boosting library XGBoost. The tree construction algorithm is executed entirely on the GPU and shows high performance with a variety of datasets and settings, including sparse input matrices. Individual boosting iterations are parallelized, combining two approaches. An interleaved approach is used for shallow trees, switching to a more conventional radix sort based approach for larger depths. We show speedups of between 3-6x using a Titan X compared to a 4 core i7 CPU, and 1.2x using a Titan X compared to 2x Xeon CPUs (24 cores). We show that it is possible to process the Higgs dataset (10 million instances, 28 features) entirely within GPU memory. The algorithm is made available as a plug-in within the XGBoost library and fully supports all XGBoost features including classification, regression and ranking tasks.
Author Comment
This is a submission to PeerJ Computer Science for review.