Predicting temperature curve based on fast kNN local linear estimation of the conditional distribution function

Introduction

Over the past decades, the volume and complexity of collected data have rapidly increased the sizes and number of covariates. This increase has created significant potential and demand for scientific and technological innovations. The increased storage capacity of information, the improvement of computers and their processing capabilities, the proliferation of surveillance systems, and the improved sensors are the technological progress that has favored the emergence of this kind of data. These are now commonly used in many fields of application such as astronomy, biology, climatology, ecology, chemistry, economics, medicine, engineering sciences, etc. The standard statistical models based on the discretization of the big data in the finite grid suffer from certain drawbacks. The first anomaly is the problem of the dimensionality curse when the size of the discretization grid is large. The second anomaly is that, with the transformation, the original data loses its characteristics such as the functionality, correlation, heteroscedasticity or the homoscedasticity of the data. In particular, the asymptotic behavior of the constructed data is related to the obtained sampling.

Recall that all the mentioned defects can substantially affect the multivariate approaches’ efficiency in big data analysis. To overcome these problems, a new approach in modern statistics called functional data analysis is developed recently. Such procedure allows the use of the natural space of the data, which permits the profit from the whole information. In this modern statistics branch, the local linearity method (LLM) estimation is widely studied. It is motivated by its small bias in the estimation processing (see Fan & Gijbels (1996) for a uni-dimensional framework, and Baìllo & Grané (2009), for the Nonparametric Functional Data Analysis (NPFDM) set up in the area of functional statistics). Barrientos-Marin, Ferraty & Vieu (2010) studied the LLM-estimation of the nonparametric operator of the Banach explanatory variable. We cite Berlinet, Elamine & Mas (2011) for an alternative LLM-estimator constructed by inverting the local variance-covariance matrix of the functional variable. Concerning the LLM-estimation of the conditional cumulative distribution function (CCDF), we point out that the first result was stated by Laksaci, Rachdi & Rahmani (2013). Later, Demongeot et al. (2014) precise the least square error of the LLM-estimator of the CCDF-model. We also point out that the previous studies utilized the kernel local linearity method. However, this paper focuses on CCDF-estimation with a new weighting approach obtained by mixing the local linear fitting to the k-Nearest Neighbors (kNN) method. Indeed, the method of kNN has been received growing attention in nonparametric functional statistics. In particular, the first results on the kNN-LLM estimation were obtained by Chikr-Elmezouar et al. (2019). They studied the conditional density estimation using the local linear method under the kNN smoothing. They established the almost complete consistency of the obtained estimator. Recently, Almanjahie et al. (2021) consider the estimation of the robust regression function using the kNN. They proved the uniform consistency on the number of the neighbor of the constructed estimator. We also mention Laksaci, Ould-Said & Rachdi (2021) for the kNN estimation of the quantile regression. For more recent results on the kNN-LLM estimation in functional statistics, we cite Rachdi et al. (2021). They treated the case when the response variable is observed with missing at random and the regressor is of functional nature.

On the other hand, the estimation based on the kNN method has more advantages than the Nadaraya–Watson algorithm (see Burba, Ferraty & Vieu (2009), for more discussion on the motivations on this approach). In this paper, we benefit from the advantages of both the kNN weighting and LLM-fitting by combining the two algorithms to provide a fast efficiency estimator for the CCDF. Specifically, we combine these approaches to construct two estimates of the CCDF and to the shortest conditional modal interval (SCMI) predictive regions using mixing functional time series. Notice also that the smoothing parameter in the k-Nearest Neighbors method is varied randomly with respect to the observations. This feature makes the applicability of these estimators very fast and accurate because the smoothing parameter is selected with respect to the nature of the data. To highlight the smoothing parameter selection issue, we compare several cross-validation rules to select the best number of the neighborhood. The superiority of both estimators in practice is emphasized by the meteorological data; specifically, the constructed estimator to predict the yearly curve of the temperature in Europe central.

This paper is structured as follows. The two kNN-LLM estimators of the CCDF are constructed in the “Method” section. We devote the section “Results and discussions” to some discussions related to our proposed estimators. The simulation study for testing the superiority of the proposed estimators is presented in the “Simulation study” section. The performance of the constructed estimator in temperature prediction using real data is conducted in the “Real-data application” section. Our conclusion and remarks for further research are presented in the last section.

Methods

In this section, we will construct two new estimators by combining the local linear approach to the kNN smoothing methods.

Results and discussions

Simulation study

In this simulation study, we propose to control the behaviour of the estimators w.r.t. the dependency degree of the data. More precisely, our aim is to compare, considering finite sample, the efficiency of the estimators $\hat{F}(y|x)$ and $\tilde{F}$(y|x). To do this, we use the fact that m-dependent variables are α-mixing and we generate n functional variables as follows.

In the first, we draw n + m − 1 independent functional variables by

$$S_j(t)=2\cos (t{W}_{1_j})+0.2W_{2_j}\text{ for }j=1,\dots ,n+m-1,$$

where W1 and W2 are uniformly distributed on [0, π/4]. Next, we simulate the m-dependent functional variables defined by:

$$X_i(t)=\sum _{j=i}^{i+m-1}{S}_j(t).$$

Discretizing the curves X_i’s on the same grid leads to the construction of 100 equispaced measurements in (0, 2π). In Fig. 1, we plotted the functional variables associated to the strong case where m = 4.

For the response variable we consider four regression models:

$$ModelM1:Y_i=5\underset{0}{\overset{2\pi }{{\displaystyle \int }}}\log ({(4-X_i(t))}^2+2)dt+{\epsilon }_i,$$

$$ModelM2:Y_i=\underset{0}{\overset{2\pi }{{\displaystyle \int }}}\exp (-X_i(t))dt+1.5\underset{0}{\overset{2\pi }{{\displaystyle \int }}}\exp (X_i(t))dt+{\epsilon }_i,$$

$$\begin{array}{ll}ModelM3:Y_i=& {\displaystyle {\int }_0^{2\pi }(5\log (}{(4-X_i(t))}^2+2))dt+{\displaystyle {\int }_0^{2\pi }\exp }(-X_i(t))dt\\ & +1.5{\delta }_i{\displaystyle {\int }_0^{2\pi }\exp }(X_i(t))dt,\end{array}$$

$$\begin{array}{ll}ModelM4:Y_i=& {\displaystyle {\int }_0^{2\pi }\exp }(-X_i(t))dt+1.5{\displaystyle {\int }_0^{2\pi }\exp }(X_i(t))dt\\ & +5{\delta }_i{\displaystyle {\int }_0^{2\pi }\log }({(4-X_i(t))}^2+2)dt,\end{array}$$

where ε_i ∼ N(0,0.25) (resp. δ_i ∼ Exp(2)). Note that, based on this models and with given X = x, the CCDF of Y is explicitly determined according to the distributions of ε_i and δ_i, which permit the determination of the theoretical CCDF, F(y|x).

Now, we specify quickly the different parameters involved in both estimators. Note that the parameters of the two estimators are the kernel K, the semi-metric d, the number k and/or l of neighbors. So, for this numerical study, we point out that we have taken a quadratic kernel supported within (0, 1) and used the L₂ metric and the numbers of neighbors k, l are chosen using the following cross-validation criterion, defined as

$$\sum _{j=1}^n(F(Y_j|X_j)-{\overline{F}}^{-j}(Y_j|X_j)),$$

where ${\overline{F}}^{-j}$ denotes the leave-one-out-curve estimate of the $\tilde{F}$ and $\hat{F}$.

The performances of the two estimates is examined by comparing their average absolute errors:

$$AE(\tilde{F})=\frac{1}{n}\sum _{i=1}^n|F(Y_i|X_i)-\tilde{F}(Y_i|X_i)|andAE(\tilde{F})=\frac{1}{n}\sum _{i=1}^n|F(Y_i|X_i)-\hat{F}(Y_i|X_i)|.$$

Thus, in order to control behaviors of both estimates w.r.t. the level of dependency, we plotted in Fig. 2, the curves of AE( $\tilde{F}$) and AE( $\hat{F}$) w.r.t. the values of m.

It can be seen that, both errors increase substantially relatively to the values of m. Furthermore, it is clear that in the models M3 and M4 (heteroscedastic case), the estimate $\hat{F}$ outperforms $\tilde{F}$, but in the models M1 and M2 (homoscedastic case), the estimate $\tilde{F}$ is significantly better than $\hat{F}$.

Real-data application

In this section, we show the applicability of the proposed estimators to a real data example. To do that, we consider the problem of predicting the monthly average temperature one year ahead. For this purpose, we consider the same data set used by Laksaci, Lemdani & Said (2011) which are available at the website https://www.met.hu/en/eghajlat/magyarorszag_eghajlata/eghajlati_adatsorok/Debrecen/adatok/napi_adatok/index.php. This data were collected by Debrecen’s station, Hungary (northern latitude 47°35′44″ and eastern longitude 21°38′43″). They are monthly measurements (1,200 months = 100 years) from 1901 to 2000. The latter can be viewed as a continuous process denoted by S_t. As noticed in the previous section, from S_t, we construct n + 1 = 100 curves (X_i(t)), i = 1,…, n + 1, where X_i denotes the average temperature curve observed during the (1 year) 12 months of the i^th year. The process (S_t) and the curves (X_i)_i are plotted in Figs. 3 and 4.

Of course, the proposed predictive interval’s efficiency is closely connected to the parameters’ choices in the estimator of the conditional distribution function. For the real data example, we compare estimators F and F in the SCMI estimation (with ζ = 0.1). For the computational study, we use the same kernel K, that is, the quadratic kernel on (0,1). The latter is adequate with this type of nonparametric approach. It is usually used in nonparametric functional statistics and incorporates the technical assumptions of the theoretical development of this kind of model. Concerning the choice of the metric d, we point out that it is closely related to the nature of the functional variable and its smoothing property. The Principal Component Analysis (PCA) metric is more suitable for this type of discontinuous functional regressors. For the choice of k (or h), we utilize the same cross-validation method used by De Gooijer & Gannoun (2000), which is based on the criterion

$$CV=\frac{1}{n}\sum _{j=1}^n\sum _{i=1}^n({\text{1}}_{\{Y_i\le Y_j\}}-{\hat{F}}^{-j}(Y_i|x_j){)}^2.$$

This criterion is optimised over the same subsets of k (or h) proposed by Rachdi & Vieu (2007), that is, {5,10,20,…,45}.

To determine the SCMI predictive interval of the whole curve of the last year (i = 100) of this sample knowing the functional covariates X₉₉, we use the first 98 curves as a training sample. Then, we predict the CCDF knowing X₉₉ by $\hat{F}$(·|X_{99 *}) and $\tilde{F}$(·|X_99*) where X_99* is the nearest curve to X₉₉ in the training sample $(Y_i^j,X_i{)}_{i=1\dots 99}$ with $Y_i^j=X_{i+1}(j)$, for each fixed month j = 1,…,12. Figure 5 displays the results. The dashed curve represents the observed data and the solid curves represent the estimated values for the two extremities of the SCMI predictive interval. We observe that the result of the $\tilde{F}$ is significantly better than the $\hat{F}$ one. In the sense that it has an average mean length (M.L = 1.23) versus M.L = 1.97 for $\hat{F}$. Of course, this gain is not surprising.

In the second illustration, we emphasize the importance of the kNN-LLM estimation of CCDF on the construction of the SCMI predictive interval by comparing the two kNN-LLM estimators, f and $\hat{F}$, to their competitive estimators constructed by the kNN-Nadraya–Watson (kNN-NW) method. More precisely, we compare the estimators $\tilde{F}$ and $\hat{F}$ to the smooth ( $\tilde{F}$₁(y|x)) and fast ( $\hat{F}$₁(y|x)) kNN-NW estimators, where

$${\widehat{F}}_1(y|x)=\frac{{\displaystyle {\sum }_{i=1}^{n}K}er\left(d(x,X_i)\text{I}{\text{H}}_k\right)H({\mathcal{l}}_l^{-1}(y-Y_i))}{{\displaystyle {\sum }_i^{n}K}er\left(\frac{d(x,X_i)}{\text{I}{\text{H}}_k}\right)}$$

and

$${\hat{F}}_1(y|x)=\frac{\sum _{i=1}^{n}Ker\left(\frac{d(x,X_i)}{{\mathrm{I}\mathrm{H}}_k}\right){\text{1}}_{\{Y_1\le y\}}}{\sum _{i=1}^{n}Ker\left(\frac{d(x,X_i)}{{\mathrm{I}\mathrm{H}}_k}\right)}.$$

Of course, in order to conduct a comprehensive comparison, we must treat the four models. For this reason, we have used the same kernel, the metric as well as the same selection method of the optimal number of neighbor k. Similar to the first illustration, this comparative analysis is performed over the 12 functional time series $(Y_i^j,X_i{)}_{i=1\dots 100}$ with $Y_i^j=X_{i+1}(j)$, for each fixed month j = 1,…,12. For each fixed j, we split (randomly) the functional time series $(Y_i^j,X_i{)}_{i=1\dots 100}$ into two parts: the learning sample (70 observations) and the test sample (30 observations). Next, we compute the SCMI predictive intervals for the curves of the testing sample using the estimators $\tilde{F}$, $\hat{F}$, $\tilde{F}$₁ and $\hat{F}$₁. The efficiency of the four models is evaluated using the Nemenyi test plots for the average mean of the interval-length (ML). The comparison results are displayed in Fig. 6.

Once again, the conclusion is without surprise. It confirms the statement mentioned in the first illustration. More precisely, the local linear approach is more accurate than the Nadaraya-Watson method. This conclusion emphasizes the superiority of the local linear over the classical kernel method, which has been shown in the multivariate statistics through the bias term. On the other hand, it should be noted that all these four functional approaches have a stratification performance in this context of predictive issues.

Conclusion

In this paper, we have studied the nonparametric estimation of the cumulative distribution function of the scalar response variable given a functional explanatory variable. Two new estimators are constructed by combining the local linear approach to the kNN smoothing methods. The first one is built by a fast algorithm based on the conditional cumulative distribution as classical regression of the indicator function. The second estimator is obtained by integration of the double kernel conditional density estimator. The latter gives a smooth estimator of the conditional cumulative distribution function. An empirical analysis is conducted to compare both estimators and their easy implementation in practice. Both artificial and real data carry out the finite sample performance of the two estimators. Undoubtedly, the present contribution highlights the conditional distribution function’s potential impact as a pivotal model in functional data analysis. It is involved in various conational nonparametric models, and its estimation is indispensable as preliminary steps to estimate numerous nonparametric functional models. For instance, we have focused on the prediction problem in this paper, and we have constructed two predictors (single prediction and region predictor). The artificial data analysis shows that both estimators have a satisfactory efficiency in different sinarios of regression data analysis, including homoscedasticity case, heteroscedasticity case, mixture models case, light-tailed or heavy-tailed conditional distribution cases. In conclusion, we can say that the functional data analysis through the conditional distribution has a significant impact in practice, and the proposed estimators of this contribution are fast, very easy to implement, and have a good performance in the prediction issues. Finally, let us note that the present contribution opens several prospects for future research. For example, it will be very interesting to compare the efficiency of our approach to other functional models such as the robust regression and the relative regression. Such models allow to reduce the sensitivity of the kNN approach to the noisy data, missing values, and the presence of outliers.

Supplemental Information