How to assess species distribution model accuracy: using internal-aspatial or external-spatial methods?

Chunrong Mi; Falk Huettmann; Yumin Guo

doi:10.7287/peerj.preprints.27257v1

How to assess species distribution model accuracy: using internal-aspatial or external-spatial methods?

Chunrong Mi^1,2,3, Falk Huettmann⁴, Yumin Guo ³

1 Chinese Academy of Science, Key Laboratory of Animal Ecology and Conservation Biology, Institute of Zoology, Beijing, China

2 University of Chinese Academy of Science, Beijing, Beijing, China

3 Beijing Forestry University, College of Nature Conservation, Beijing, China

4 University of Alaska Fairbanks (UAF), EWHALE Lab- Department of Biology and Wildlife, Institute of Arctic Biology, Fairbanks, Alaska, USA

DOI: 10.7287/peerj.preprints.27257v1

Published: 2018-10-07
Accepted: 2018-10-07

Subject Areas: Biodiversity, Biogeography, Conservation Biology, Ecology, Zoology
Keywords: Species distribution models (SDMs), Aspatial and Spatial model accuracy metrics, TSS, Kappa, AUC, Random Forest, TreeNet (Boosting), Hooded Crane (Grus monacha)

Copyright: © 2018 Mi et al.
Licence: This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ Preprints) and either DOI or URL of the article must be cited.

Cite this article: Mi C, Huettmann F, Guo Y. 2018. How to assess species distribution model accuracy: using internal-aspatial or external-spatial methods? PeerJ Preprints 6:e27257v1 https://doi.org/10.7287/peerj.preprints.27257v1

Abstract

Species distribution models (SDMs) have become an increasingly important tool in ecology, biogeography, evolution and, more recently, in conservation management, landscape planning and climate change research. The assessment of their predictive accuracy is one fundamental issue in the development and application of SDMs. Accuracy assessments for models should have a close connection to the intended use of the model. However, we found that the common evaluation method (we named internal-aspatial) usually ignored how the spatial prediction map actually looks like, and achieves for the real-world species distribution and for application. Therefore, in this research we proposed a spatial method to evaluate model performance by assessing how the prediction maps look like (we named external-spatial). We took Hooded Crane (Grus monacha) as a case, in this research, to compare these two methods (internal-aspatial and external-spatial) performance. Both of the two methods were expressed with three commonly used SDM evaluation criteria (AUC, Kappa and TSS). In addition, model accuracy was also assessed via evaluating the prediction maps with knowledge of the study species and alternative occurrence data assistance. We used two popular data mining algorithms (Random Forest and TreeNet) and ran 8 experiments using 1, 3, 5, 8, 11, 21, 29 and 78 predictors, allowing to develop overall 16 models for this assessment. Results indicated that AUC had a significant linear relationship with Kappa and TSS. Both of interal-aspatial and external-spatial methods could get higher AUC values and they were close. This indicated that internal-aspatial model assessments can serve as powerful assessment-aspatiual metrics without the need of secondary data even! However, internal-aspatial, external-spatial, prediction map evaluation and alternative occurrence data could not distinguish well models with different sets of predictors. This is the first time the concept of spatial assessment criteria is expressed and assessed. Overall, we hope to see more study on meaningful spatial criteria and proposed more and better methods to evaluate SDMs and distribution map in the future.