A retrospective approach for evaluating ecological niche modeling transferability over time: the case of Mexican endemic rodents

Introduction

Climate change is significantly impacting biodiversity, causing shifts in species abundance and distribution that lead to extensive reshuffling of biotas (Parmesan & Yohe, 2003; Pacifici et al., 2015; Peterson et al., 2015; MacLean et al., 2018; Widick & Bean, 2019; Habibullah et al., 2022; Jaroszynska et al., 2023). In response to these dynamic changes, ecological niche modeling (ENM) has emerged as a valuable tool for analyzing and predicting species’ geographic distributions under various climatic scenarios, both past and future. Grounded in ecological niche theory, ENM integrates methodologies from multiple disciplines, including informatics, geographic information systems (GIS), and statistics (Austin, 2007; Soberón & Nakamura, 2009; Wiens et al., 2009; Sillero et al., 2021). This correlative approach utilizes georeferenced species occurrence data and environmental predictor variables to model the ecological niche of species, and projecting this information onto geographic space to generate a map that is commonly referred to as the species’ potential distribution (Barbet-Massin & Jetz, 2014; Elith et al., 2006; Peterson, 2011; Soley-Guardia, Alvarado-Serrano & Anderson, 2024).

Several methods for constructing niche models have evolved to address different types of occurrence data: presence-only (e.g., Bioclim, environmental distances), presence-absence (e.g., generalized linear models (GLM), generalized additive models (GAM)), presence-pseudoabsence (e.g., Genetic Algorithm for Rule-set Prediction (GARP)), and presence-background (e.g., Ecological Niche Factor Analysis (ENFA), Maxent) (Phillips, 2008; Barbet-Massin et al., 2012; Barbet-Massin & Jetz, 2014; Warton & Aarts, 2013; Fan et al., 2018; Qiao et al., 2019; Sillero et al., 2023). Presence-absence methods are considered more robust when unequivocal absence data are available, as they help identify unsuitable areas that might be misclassified by presence-only methods (Brotons et al., 2004; Golicher et al., 2012). However, reliable absence data are rare, leading to the predominance of presence-only, presence-pseudoabsence, and presence-background algorithms (Vaz, Cunha & Nabout, 2015; Soley-Guardia, Alvarado-Serrano & Anderson, 2024; Sillero et al., 2021).

ENM is extensively used to evaluate the potential impacts of climatic changes by projecting modeled niches under current conditions onto past or future scenarios, a concept known as model transferability (Pearson & Dawson, 2003; Peterson, Martínez-Meyer & González-Salazar, 2004; Thomas et al., 2004; Waltari & Guralnick, 2009; Heikkinen, Marmion & Luoto, 2012; Garcia et al., 2016; Zhu, Fan & Peterson, 2021; Sillero et al., 2021). The success of this transferability lies on three critical assumptions: (1) the ecological niche remains stable during the period of transfer (Soberón & Nakamura, 2009); (2) the relationship between environmental variables and species remains constant during climatic changes (Hijmans & Graham, 2006; Wiens et al., 2009); and (3) the species is in equilibrium with the environment in the calibration scenario, meaning they occupy the available suitable areas accessible to them (Sequeira et al., 2018; Yates et al., 2018; Werkowska et al., 2017). These premises are critical as they assume that the current relationship between the species and the environment is optimal, and transferability will be defined by how closely the model conforms to this relationship.

A major challenge in model transferability is the presence of non-analog climates—environmental conditions in the projected climatic scenario that are absent in the calibration scenario (Sequeira et al., 2018). Algorithms respond idiosyncratically to these conditions due to their programmatic features, frequently producing disparate results (Pearson, 2006; Araújo & Peterson, 2012; Essl et al., 2023). This issue is particularly problematic when projecting ecological niches into future climates, where no empirical data are available to validate algorithm performance. Consequently, a common approach for evaluating algorithm effectiveness in climate change studies involves projecting models between two historical periods for which data are available, such as from the recent past to the present (Rubidge et al., 2011; Piirainen et al., 2023).

Extensive research has been conducted to compare algorithm performance across spatial and temporal transferences, utilizing both real and virtual species (Prasad, Iverson & Liaw, 2006; Hijmans & Graham, 2006; Kharouba, Algar & Kerr, 2009; Dobrowski et al., 2011; Rubidge et al., 2011; Moreno-Amat et al., 2015; García-Callejas & Araújo, 2016). These studies reveal that results can vary greatly due to factors such as species traits, biotic interactions, data completeness, and climatic dissimilarities. Such variations complicate the identification of the specific impacts that algorithm choices have on the outcomes (Yates et al., 2018; Merow et al., 2014).

In this study, we evaluated the performance of seven popular modeling algorithms—Bioclim, GAM, GLM, boosted regression trees (BRT), Maxent, random forest (RF), and support vector machine (SVM)—in transferring niche models of Mexican endemic rodents from the mid-20th century to the late-20th/early 21st centuries and vice versa. We selected Mexican endemic rodents due to their restricted and relatively well-known distributions, and because we do not expect significant climatic niche evolution over this period (Martínez-Meyer, Peterson & Hargrove, 2004). We expected that species with poor data quality (e.g., scarce or biased) would produce poor results regardless of the algorithm used. Conversely, for well-sampled species, we hypothesized that algorithms capable of modeling biologically meaningful response curves (e.g., bell-shaped responses to temperature), such as Maxent or SVM, would outperform simpler algorithms like Bioclim or GLM.

Materials and Methods

Model transferences

The logical procedure to evaluate the capacity of algorithms to transfer niche models across different temporal scenarios involves calibrating a niche model in period 1 and transferring it onto the climatic scenario of period 2, then validating the transference with occurrences from period 2, or calibrating a niche model in period 2 and comparing the two maps (Hijmans & Graham, 2006). However, disparities in the number or environmental distribution of occurrences between periods can affect observed differences between the resulting maps, making it difficult to attribute such differences to algorithm performance alone. To address this issue, we implemented a cross-temporal approach to identify species with similar occurrence data structures in both time periods.

First, we calibrated a model using occurrences and climatic layers from period 1 (“auto1”); then, we generated a second model using the climatic layers from period 1 with occurrences from period 2 (“cross1”). Next, we calibrated a model using occurrences from period 2 with the climatic surfaces from that period (“auto2”) and another model using occurrences from period 1 with the climatic surfaces from period 2 (“cross2”). All resulting maps were converted into binary format, and we compared “auto1” with “cross1” and “auto2” with “cross2” geographically (see next section). If the overall similarity between the two pairs of maps was less than 70%, it indicated significant differences in the number or distribution of occurrences between the time periods, which could hinder the ability of algorithms to transfer models across temporal scenarios. We repeated this procedure for all species and algorithms. Species for which the similarity value was below 70% were excluded from further analyses. Species with suitable datasets for analysis were classified as “control” species, while those without were labeled “without control.” For control species, we proceeded with transferring models from period 1 to period 2 (forecast) and vice versa (hindcast) and subsequently evaluated model performance (Fig. 1).

Results

Niche models and model transferences

Binomial tests revealed that models calibrated with occurrences and climatic layers from the same period (auto1 and auto2) significantly deviated from random expectations for most species, indicating reliable model accuracy. Exceptions included GLM models for Callospermophilus madrensis, Dasyprocta mexicana, and Dipodomys phillipsii in T1 (1950–1979), and Dipodomys phillipsii, Neotamias durangae, Neotoma goldmani, and Oryzomys guerrerensis in T2 (1980–2009); a BRT model for Pappogeomys bulleri in T2, and a Maxent model for Dasyprocta mexicana in T2 (Supplemental Material, Table S4).

We found significant differences between algorithms for transferring models from T1 to T2 and vice versa (H = 78.75 and H = 79.08, respectively, both p < 0.01). RF consistently showed the highest mean TSS ( $\overline{\mathrm{x}}$ = 0.83 ± 0.08) in forecasting, while Maxent in hindcasting ( $\overline{\mathrm{x}}$ = 0.82 ± 0.10). Notably, Maxent showed significantly higher TSS values for hindcasting than forecasting (w = 435, p = 0.002). Conversely, BRT, RF, and Bioclim showed no significant directional differences. Bioclim recorded the lowest in both forecasting ( $\overline{\mathrm{x}}$ = 0.50 ± 0.20) and hindcasting ( $\overline{\mathrm{x}}$ = 0.52 ± 0.21).

Significant differences between algorithms were also evident in OI (H = 27.27, p = 0.014 in hinadcasting and H = 15.93, p < 0.01 in forecasting), with Bioclim ( $\overline{\mathrm{x}}$ = 0.52 ± 0.21) and GAM ( $\overline{\mathrm{x}}$ = 0.65 ± 0.18) showing the greatest variation in forecasting, and Maxent also displaying significant directional differences (w = 382, p < 0.01). RF maintained consistent performance across both directions (forecasting: $\overline{\mathrm{x}}$ = 0.83 ± 0.08; hindcasting: $\overline{\mathrm{x}}$ = 0.80 ± 0.09) (Fig. 2; Supplemental Material, Table S5).

The FPR varied significantly among algorithms in both forecasting and hindcasting (H = 92.237 and H = 77.102, both p < 0.01), with Bioclim showing the highest rates both in forecasting ( $\overline{\mathrm{x}}$ = 0.47 ± 0.22) and hindcasting ( $\overline{\mathrm{x}}$ = 0.47 ± 0.22), and RF the lowest (forecasting: $\overline{\mathrm{x}}$ = 0.09 ± 0.09; hindcasting: $\overline{\mathrm{x}}$ = 0.10 ± 0.08). Maxent showed significant differences in FPR between directions (w = 1062, p = 0.003). The FNR also showed significant differences between algorithms for hindcasting (H = 51.171, p < 0.01) and forecasting (H = 65.92, p < 0.01), where GAM recorded the highest values (Fig. 2; Supplemental Material, Table S5).

Correlation analyses

Our correlation analyses between the number of occurrence records and TSS scores showed weak and non-significant positive relationships for most algorithms (R² < 0.1, p > 0.05). Exceptions were Bioclim, GAM, and GLM in hindcasting, with GLM also showing a marginal significance in forecasting (Fig. 3). In general, species with larger sample sizes achieved higher TSS scores.

Discussion

Ecological niche model transfers are the cornerstone for analyzing species’ distributional impacts due to climate change, yet their reliability is often overlooked and remains untested in most cases. The seven algorithms tested in this study exhibited varying degrees of robustness in their ability to accurately transfer niche models across time, introducing additional challenges in our efforts to anticipate or reconstruct the geographic consequences of climate change. In particular, RF, Maxent, BRT, and SVM consistently performed well, while Bioclim showed the poorest performance. Notably, Maxent exhibited significant sensitivity to the direction of transfer, with marked differences between hindcasting and forecasting. Our results are consistent with previous studies that have observed variations in algorithm performance when transferring niche models to different climatic scenarios (Moreno-Amat et al., 2015; Beaumont et al., 2016; Bell & Schlaepfer, 2016; García-Callejas & Araújo, 2016; Liang et al., 2018; Qiao et al., 2019; Heikkinen, Marmion & Luoto, 2012). However, to our knowledge, no prior studies have examined how occurrence data quality affects algorithm performance in both forecasting and hindcasting.

The presence of non-analog climates is known to affect model transferability between time periods (Sequeira et al., 2018; Charney et al., 2021; Essl et al., 2023). Climatic combinations outside the calibration scenario challenge all algorithms, particularly those with limited extrapolation capacity, such as Bioclim (Qiao et al., 2019). To assess the influence of non-analog climates on algorithm performance, we conducted a Mobility-Oriented Parity (MOP) analysis (Owens et al., 2013), which quantifies the multidimensional similarity between two climatic scenarios (calibration and transfer) and maps areas requiring strict extrapolation, with the smop package (Osorio-Olvera & Contreras-Díaz, 2024) in R. Our results indicate that the areas with dissimilar climatic combinations—and thus where strict extrapolation is needed—are limited across Mexico: from T1 to T2, they occupy 0.58% of the country, and from T2 to T1, 1.7% (Supplemental Material, Fig. S1). Therefore, non-analog climates do not explain most of the observed variation in algorithm performance.

Another crucial factor affecting model performance is data quality, specifically the representation of the environmental combinations that define a species’ ecological niche (van Proosdij et al., 2016; Jiménez-Valverde, 2020). Low-quality data may result from insufficient sampling or environmental bias (Wang & Jackson, 2023). We initially hypothesized that species with fewer occurrences would show poor transferability across all algorithms, as small sample sizes often lead to an incomplete ecological niche characterization. However, our results indicate that sample size impacts algorithms differently: while sample size minimally affected transferability for RF, BRT, and SVM, it significantly impacted Bioclim, GLM, GAM, and, to a lesser extent, Maxent. These findings align with previous studies where Maxent outperformed Bioclim (Hernandez et al., 2006) and GAM (Wang & Jackson, 2023) under small sample sizes.

Sample size also explains performance differences among algorithms in forecasting and hindcasting. In our study, Maxent had the greatest directional difference, showing greater robustness during hindcasting than forecasting (Fig. 2). Notably, Maxent’s sensitivity to sample size was evident only in forecasting (Fig. 3), suggesting that small sample sizes more adversely affect its transferability than other algorithms like RF and BRT. This disparity in performance between Maxent and RF is consistent with Wang & Jackson’s (2023) findings, who recommend RF for small sample sizes.

Modeling algorithm robustness with varying data qualities appears to hinge on their ability to accurately characterize the geometrical complexity of the ecological niche (García-Callejas & Araújo, 2016)—the structural characteristics of the boundary between suitable and unsuitable conditions in environmental space. For species with well-defined niches, like those specialized to specific environments, robust models are generally easier to obtain with less occurrences (Hallman & Robinson, 2020). In contrast, species with broader environmental preferences require more unbiased samples to accurately characterize their more complex niche boundaries. Consequently, some algorithms manage boundary complexity better than others, especially with small sample sizes.

Our research focused on Mexican endemic rodents, most of which are habitat specialists with narrow distributions, although there are some exceptions, such as Peromyscus difficilis (Ceballos & Oliva, 2005). Therefore, we expected that small sample sizes would be sufficient to produce reliable models for most algorithms. However, our findings showed this was not the case, indicating that insufficient sample sizes negatively impacted model robustness and, consequently, temporal transferability. This effect is likely to be even more pronounced for species with broad ecological requirements, regardless of the geographic location or taxonomic group (Moudrý et al., 2024).

We found that Bioclim had the poorest performance in transferring niche models across temporal climatic scenarios. Bioclim, a simple environmental envelope model based on the range of values from occurrence records in the predictor variables (Nix & Busby, 1986), is highly sensitive to extreme values and the number of predictors (Beaumont, Hughes & Poulsen, 2005). Additionally, Bioclim’s quadrangular representation of the ecological niche in environmental space limits its ability to model complex niche geometries. A similar issue may arise with GLM, particularly with small sample sizes (Guisan, Edwards & Hastie, 2002).

In contrast, RF and Maxent exhibited the greatest transfer capacity, followed closely by BRT and SVM. RF also had the most consistent performance between transfer directions. This algorithm has proven robust for transferring models with both virtual (García-Callejas & Araújo, 2016) and real (Mi et al., 2017) species, with good interpolation performance (Bell & Schlaepfer, 2016; Liang et al., 2018), and relatively little overprediction (Mi et al., 2017). However, this robustness comes at the cost of overfitting, which sometimes limits its extrapolation capacity beyond the calibration range (Heikkinen, Marmion & Luoto, 2012). Maxent, on the other hand, is less prone to overfitting, particularly when parameterized ad hoc for specific species (Merow, Smith & Silander, 2013). In summary, the four machine-learning algorithms generally outperformed the two regression-based and climatic envelope algorithms. However, as highlighted in numerous comparative studies, there is no "silver bullet" algorithm that consistently performs best across all data structures (Qiao, Soberón & Peterson, 2015). This is even more evident in model transferability, where algorithm weaknesses are amplified (Pearson, 2006; Moreno-Amat et al., 2015).

A final note of caution is that while our analyses were designed to evaluate algorithm performance for temporal transferability using Mexican endemic rodents as the model system, a potential source of error lies in the continuously updated taxonomy of these species. Our analyses were based on the latest revision of Mexican mammals (Ramírez-Pulido et al., 2014); however, recent proposals suggest species separation for Peromyscus melanophrys, P. furvus, P. levipes, P. zarhynchus, and Osgoogomys banderanus (Lorenzo et al., 2016; Almendra et al., 2018; Cruz-Gómez et al., 2021; Bradley et al., 2022). These taxonomic changes could affect the quantity and spatial structure of occurrences for these species, potentially impacting model performance (Soley-Guardia, Alvarado-Serrano & Anderson, 2024).

Conclusions

Ecological niche modeling is often used to transfer models across temporal scenarios for climate change analysis. However, the suitability of the species for such transfers and the robustness of the chosen algorithms are often overlooked. Our results highlights that the performance of these algorithms, and consequently the reliability of temporal transfers, is primarily influenced by the quality of data. Sample size is a key element for the effectiveness of model transfers since small sample sizes reduce the capacity of some algorithms more than others. Indeed, an algorithm may yield different results for the same species when transferring models to past vs future scenarios, indicating that not all species and algorithms are equally suited for transferring models across temporal scenarios. Among the algorithms evaluated, those capable of modeling complex ecological boundaries with minimal overfitting—such as random forest, Maxent, and boosted regression trees—consistently outperformed the simpler algorithms Bioclim and GLM. Consequently, we strongly recommend a careful assessment of both species and algorithms before proceeding with temporal transfers. In this regard, the retrospective cross-temporal approach presented here offers a valuable alternative.

Supplemental Information