How to assess species distribution model accuracy: using internal-aspatial or external-spatial methods?
Author and article information
Abstract
Species distribution models (SDMs) have become an increasingly important tool in ecology, biogeography, evolution and, more recently, in conservation management, landscape planning and climate change research. The assessment of their predictive accuracy is one fundamental issue in the development and application of SDMs. Accuracy assessments for models should have a close connection to the intended use of the model. However, we found that the common evaluation method (we named internal-aspatial) usually ignored how the spatial prediction map actually looks like, and achieves for the real-world species distribution and for application. Therefore, in this research we proposed a spatial method to evaluate model performance by assessing how the prediction maps look like (we named external-spatial). We took Hooded Crane (Grus monacha) as a case, in this research, to compare these two methods (internal-aspatial and external-spatial) performance. Both of the two methods were expressed with three commonly used SDM evaluation criteria (AUC, Kappa and TSS). In addition, model accuracy was also assessed via evaluating the prediction maps with knowledge of the study species and alternative occurrence data assistance. We used two popular data mining algorithms (Random Forest and TreeNet) and ran 8 experiments using 1, 3, 5, 8, 11, 21, 29 and 78 predictors, allowing to develop overall 16 models for this assessment. Results indicated that AUC had a significant linear relationship with Kappa and TSS. Both of interal-aspatial and external-spatial methods could get higher AUC values and they were close. This indicated that internal-aspatial model assessments can serve as powerful assessment-aspatiual metrics without the need of secondary data even! However, internal-aspatial, external-spatial, prediction map evaluation and alternative occurrence data could not distinguish well models with different sets of predictors. This is the first time the concept of spatial assessment criteria is expressed and assessed. Overall, we hope to see more study on meaningful spatial criteria and proposed more and better methods to evaluate SDMs and distribution map in the future.
Cite this as
2018. How to assess species distribution model accuracy: using internal-aspatial or external-spatial methods? PeerJ Preprints 6:e27257v1 https://doi.org/10.7287/peerj.preprints.27257v1Author comment
This is a submission to PeerJ for review.
Sections
Supplemental Information
Supplement S1 Location of presence point
Supplement S1 Location of presence point
Additional Information
Competing Interests
The authors declare that they have no competing interests.
Author Contributions
Chunrong Mi conceived and designed the experiments, performed the experiments, analyzed the data, contributed reagents/materials/analysis tools, prepared figures and/or tables, authored or reviewed drafts of the paper, approved the final draft.
Falk Huettmann contributed reagents/materials/analysis tools, approved the final draft.
Yumin Guo contributed reagents/materials/analysis tools, authored or reviewed drafts of the paper, approved the final draft.
Data Deposition
The following information was supplied regarding data availability:
The raw data has been provided as a Supplemental File
Funding
This research is funded by the National Natural Science Foundation of China (project no. 31570532), the State Forestry Administration of China. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
