Substation equipment temperature prediction based on multivariate information fusion and deep learning network

Correlation analysis

Two functions of autocorrelation function and partial autocorrelation function are adopted to analyze correlation. The autocorrelation functon and partial autocorrelation function are described as follows. (1) As is known, autocorrelation belongs to sequence correlation, which expresses the cross-correlation between the sequence and itself at different moments (Chachlakis et al., 2021). The autocorrelation coefficient of the time series is denoted as ACF, that is autocorrelation function. This article quantitatively describes the lag autocorrelation of substation equipment temperature time series by calculating ACF value. ACF is expressed as $\stackrel{\wedge }{{\rho }_k}$ in formula Eq. (1): (1)${\displaystyle \stackrel{\wedge }{{\rho }_k}=\frac{\sum _{t=1}^{n-k}\left(Z_t-\stackrel{-}{Z}\right)\left(Z_{t+k}-\stackrel{-}{Z}\right)}{\sum _{t=1}^n{\left(Z_t-\stackrel{-}{Z}\right)}^2}}$

where, Z_t is the equipment temperature at time t, Z_t+k is the equipment temperature at time t + k, $\stackrel{-}{Z}$ is the average value of equipment temperature. (2) Partial autocorrelation is the relationship summary between the time series observation after eliminating interference and the previous time step observation (Mestre et al., 2021). That is, consider the correlation after removing the influence of intervention variables Z_t+1, Z_t+2, Z_t+3, … with common linear dependence from Z_t and Z_t+k, namely, under the condition of observation Z_t+1, the autocorrelation state of Z_t and Z_t+k so on. Partial autocorrelation function (PACF) is expressed as P_k in formula Eq. (2): (2)${\displaystyle P_k=\frac{Cov\left[\left(Z_t-\stackrel{\wedge }{Z_t}\right),\left(Z_{t+k}-\stackrel{\wedge }{Z_{t+k}}\right)\right]}{\sqrt{Var\left(Z_t-\stackrel{\wedge }{Z_t}\right)}\sqrt{Var\left(Z_{t+k}-\stackrel{\wedge }{Z_{t+k}}\right)}}}$

where, Cov refers to the covariance at moment t, Var refers to sample variance, $\stackrel{\wedge }{Z_t}$ is sample estimation at moment t, and $\stackrel{\wedge }{Z_{t+k}}$ is sample estimation at moment t + k.

PCA

Principal component analysis(PCA) is a data dimension reduction method that is widely applied in various fields (Cao, Sun & Zhao, 2022), which has the functions of simplifying operation, removing data noise and discovering hidden related variables. Therefore, PCA is selected to screen the input features. By calculating cumulative contribution rate of the input features, the first few important features are selected from multiple features as the principal components to reduce the input dimension and improve the convergence speed.

The main idea of PCA is to relinearly combine p-dimensional linearly related features and map them into k-dimensional linearly independent features (k < p). The reacquired k-dimensional features are principal components, which can represent the information of the original features to the greatest extent.

It is assumed that it has pfeatures, and each feature has n observation values, then the initial data matrix C can be obtained.

The implementation process of PAC method is realized by the following six steps:

(1) The original p characteristics are standardized to obtain the standardized feature variables. (4)${\displaystyle y_j=\frac{s_j-{\mu }_j}{s_j},j=1,2,\dots ,p}$

where, ${\mu }_j=\frac{1}{n}{\sum }_{i=1}^n{c}_{ij},s_j=\sqrt{\frac{1}{n}{\sum }_{i=1}^n{\left(c_{ij}-{\mu }_j\right)}^2}$.

(2) Standardize each feature element to obtain the corresponding data matrix W. (5)${\displaystyle W=\left[\begin{array}{c}{w}_{11}{w}_{12}\cdots w_{1p}\hfill \\ w_{21}{w}_{22}\cdots w_{2p}\hfill \\ ⋮⋮⋮⋮\hfill \\ w_{n1}{w}_{n2}\cdots w_{np}\hfill \end{array}\right]}$

where, $w_{ij}=\frac{c_{ij}-{\mu }_{ij}}{s_j},$i =1 , 2, …, n; j =1 , 2, …, p.

(3) According to the matrix W, the correlation coefficient matrix $R={\left(r_{ij}\right)}_{p\times p}$of W is calculated. Where, $r_{ij}=\frac{{\sum }_{t=1}^n{w}_{t1}{w}_{tj}}{n-1},i,$j =1 , 2, …, p.

(4) Calculate the eigenvalues of matrix R andsort them in descending order λ₁ ≥ λ₂ ≥ ⋯ ≥ λ_p, and the standard orthogonalization eigenvector corresponding to each eigenvalue is calculated u₁, u₂, …, u_p, where, ${\mu }_j={\left[{\mu }_{1j},{\mu }_{2j},\dots ,{\mu }_{pj}\right]}^T,$j =1 , 2, …, p.

(5) p new feature vectors are computed with the original p standard orthogonal feature elements, that is, (6)${\displaystyle \left\{\begin{array}{c}{N}_1=u_{11}{y}_1+u_{21}{y}_2+\cdots +u_{p1}{y}_p\hfill \\ N_2=u_{12}{y}_1+u_{22}{y}_2+\cdots +u_{p2}{y}_p\hfill \\ ⋮\hfill \\ N_p=u_{1p}{y}_1+u_{2p}{y}_2+\cdots +u_{pp}{y}_p\hfill \end{array}\right.}$

where, N₁ refers to the first principal component; N₂ is the second principal component; N_p is the p − th principal component.

(6) The contribution rate and cumulative contribution rate of each principal component are calculated, and the calculation formula is shown in formula Eq. (7) and formula Eq. (8) respectively. (7)${\displaystyle N_j=\frac{{\lambda }_j}{\sum _{t=1}^p{\lambda }_t},j=1,2,\dots ,p}$ (8)${\displaystyle {\eta }_i=\frac{\sum _{t=1}^i{\lambda }_t}{\sum _{t=1}^p{\lambda }_t}\times 100\text{\%},i=1,2,\dots ,p.}$

Among them, N_j is the contribution rate of the j − th principal component; η_i is the cumulative contribution rate of the first i principal components.

CNN

CNN is the abbreviation of convolutional neural network, which is a variant of multilayer perceptron (MLP), and it was developed by biologists Huber and Wiesel in their early research on cat visual cortex (Aslan et al., 2022). Figure 1 shows the structure of CNN networks. The structure of CNN is described in order, including input layer, convolution layer, activation layer, pool layer, full connection layer and output layer. The convolution layer is the core structure of CNN model, which is usually 1 × 1 matrix, 3 × 3 matrix and 5 × 5 matrix. The weights of neurons on the same feature mapping plane in CNN can be shared locally. Therefore, CNN network supports parallel learning, which can greatly improve the calculation speed and model prediction efficiency. The unique structure of CNN has great advantages in the fields of machine learning, deep learning and prediction field, which is the most widely used depth feature extraction method.

GRU network

Gate recurrent unit (GRU) is a special network structure in neural network (Ansari, Bartoš & Lee, 2022), which has only two gate structures of reset gate and update gate, is simpler than the three gate structure of LSTM network and has good prediction effect. These two gating vectors can determine which data can be used as the final output. The basic structure of GRU is shown in Fig. 2.

In Fig. 2, x_t refers to the input data, that is, the high-dimensional feature vector, h_t−1 refers to the output data of the previous layer, and h_t refers to the output data of the current layer. r_t and z_t are the outputs of reset gate and update gate, and k_t is the candidate set. σ and tanh are sigmoid activation function and tanh activation function. The mathematical description of GRU is shown in formula Eq. (9). (9)${\displaystyle \left\{\begin{array}{c}{z}_t=\sigma \left(W_z\cdot \left[h_{t-1},x_t\right]+b_z\right)\hfill \\ r_t=\sigma \left(W_r\cdot \left[h_{t-1},x_t\right]+b_r\right)\hfill \\ k_t=tanh\left(W_k\cdot \left[r_t\cdot h_{t-1},x_t\right]+b_k\right)\hfill \\ h_t=\left(1-z_t\right)\cdot h_{t-1}+z_t\cdot k_t\hfill \end{array}.\right.}$

Temperature data acquisition of substation equipment

The research object of this article is primary equipment of main transformer in a substation from Taizhou City, Zhejiang Province. The substation adopts the intelligent inspection system. The temperature of each monitoring point for the equipment is measured by the infrared camera and stored in the form of multi-dimensional intelligent inspection history curve analysis report, including the substation equipment temperature monitoring serial number, organization, measurement position, inspection time, measured value and description (describe the equipment status, whether it is normal or not).

Substation equipment monitoring points are distributed at 110 kV side and 220 kV side, which includes four parts of bushing, conservator, heat sink and panorama. In this article, the data at 220 kV side are selected for the experiment. Monitoring point information is shown in Table 1.

The infrared camera of the equipment is set to monitor once every hour, and the data acquisition time is 15 months from December 11, 2020 to March 10, 2022. However, there are power outage maintenance and bad points in the monitoring process. Therefore, this article adopts the method of direct elimination, and finally obtains 3,906 effective experimental data.

The selection of data will directly affect the effectiveness of the prediction model. According to the typical seasonal characteristics of temperature, this article selects the data of the first 12 months in the experimental data for training, that is, 3,086 data from December 11, 2020 to December 10, 2021, and 820 data from December 11, 2021 to March 10, 2022.

For the primary equipment of main transformer in substation, the temperature of bushing has the greatest impact on the equipment, so the temperature of A contact from bushing phase is selected as the prediction target for the experiment. Figure 4 shows the thermal imaging diagram of phase A contact at 220kV side bushing on October 1, 2020.

Feature engineering

A. Data Analysis and Feature Selection

Figure 5 shows the temperature data of all monitoring points on 220 kV side from the primary equipment of No. 2 main transformer, and two conclusions can be drawn: (1) With seasonal changes, the equipment temperature also changes significantly. The corresponding performance of the same monitoring point in different seasons is different. The average temperature in winter is about 20 °C and the average temperature in summer is about 50 °C. It can be seen that there is obvious correlation between equipment temperature and environmental factors. Therefore, when predicting the equipment temperature, it is necessary to consider the ambient temperature factor. (2) The temperature trend of different monitoring points for the same equipment shows obvious consistency, which means that there is typical linear correlation between the temperature of equipment space correlation monitoring points.points for the same equipment shows obvious consistency, which means that there is typical linear correlation between the temperature of equipment space correlation monitoring points.

In addition, Fig. 6 shows the correlation analysis results of historical temperature time series from phase A contact of bushing. According to the analysis results of autocorrelation and partial autocorrelation, it can be determined that the temperature time series of phase A contact for bushing is an unstable series. From ACF and PACF between the temperature time series and its first-order difference series, it can be seen that they are trailing, indicating that the historical temperature of substation equipment has strong correlation, and the influence of past time decreases gradually with the passage of time.

In summary, the substation equipment temperature has typical seasonality, periodicity and instability. Therefore, when predicting the equipment temperature, this article determines that the feature vector of multivariate information fusion is composed of the characteristics of ambient, time and space, which is recorded as $MIFFV=\left[A,T,S\right]$, where, A refers to the ambient feature, T refers to the time feature and Srefers to the space feature. The specific description is as follows:

1. Ambient feature. In Part A, it is found that the substation equipment temperature is greatly affected by the ambient temperature. Therefore, the weather conditions are taken as the ambient feature in this article, which are recorded as $A=\left[A_1,A_2,A_3,\dots \dots ,A_{d1}\right]$, d1 is the dimension of ambient feature. In addition, considering that Zhejiang Province belongs to a typical subtropical monsoon climate, with low temperature and little rain in winter, prevailing northwest wind, high temperature and rain in summer, prevailing southeast wind and muggy, this article determines to take real-time weather temperature and humidity as ambient characteristics to form the ambient feature vector($A=\left[A_1,A_2\right]$, that is, set d1 = 2). Because the temperature of substation equipment is set to be collected every hour, in order to obtain ambient characteristics, Java programming is used to collect weather conditions every hour through the weather interface of Juhe API (website: http://www.juhe.cn), and two columns of weather temperature and humidity are selected as ambient characteristics.

2. Time feature. According to the working experience of substation operation and maintenance personnel and the autocorrelation and partial autocorrelation analysis results, the time series of substation equipment temperature has strong lag correlation. Therefore, the historical temperature time series of substation equipment is selected as the time feature vector, which is recorded as $T=\left[T_1,T_2,\dots \dots ,T_{d2}\right]$. Although the lag correlation is relatively large, considering that this article adopts the feature vector of multi information fusion, in order to avoid the inclination of the feature vector in the time feature due to too many time features, the equipment temperature values of the past three times are selected as the time feature, that is d2 = 3.

3. Space feature. The primary equipment of No. 2 main transformer is taken as the research object. For such substation equipment, including 110 kV and 220 kV sides, and both sides are relatively independent, the article selects the temperature of phase A contact for bushing on 220 kV side as the prediction target for the experiment. Therefore the temperature of all monitoring points with space correlation with phase A contact for bushing is composed of space feature vector, which is recorded as $S=\left[S_1,S_2,\dots \dots ,S_{d3}\right]$. The names of all monitoring points are recorded in Table 1. There are seven infrared temperature monitoring points on the 220 kV side, that is, in addition to the bushing phase A contact, there are six spatial correlation monitoring points, namely bushing B-phase contact, bushing c-phase contact, conservator, No. 1 heat sink, No. 2 heat sink temperature and 220 kV side panoramic temperature. Therefore, set d3 = 6.

B. Feature extraction—reduced feature vector based on PCA

There are 11 characteristics in MIFFV of multivariate information fusion composed of three aspects of ambient, time and space, which can comprehensively characterize the temperature. While, too much input data can not improve prediction accuracy, but it is easier to produce information redundancy. Therefore, PCA is adopted to reduce the dimension. In general, the eigenvector composed of eigenvalues with cumulative contribution rate of 85%–95% is used as the principal component. Through many experiments, it is verified that the effect of the eigenvalue prediction is the best when the cumulative contribution rate reaches 98%, therefore, the principal components are taken as the reduced feature vector (denoted as RFV) under 98% cumulative contribution rate in this article. In the experimental process, PCA dimensionality reduction mapping matrix is shown in Fig. 7, and the feature contribution rate pie chart is shown in Fig. 8.

Temperature prediction of substation equipment

Deep learning network based on CNN and GRU (CNN-GRU) is adopted to predict the phase A contact of bushing, where, CNN filter size is 10; the training cycle is 24 times per round, 60 rounds in total, and the total number of iterations is 1,440; the learning rate is 0.005 and the error threshold is 0.001.

The prediction results for test set based on CNN-GRU are shown in Fig. 9, and the testing relative error is shown in Fig. 9. From the above results, it can be summed up that the temperature prediction effect of bushing phase A contact based on CNN-GRU network is good, the relative error remains between ±0.2, and there is a relatively large error between the sample 450 and 500 in the test set from Fig. 10. The results show that because too many missing points and bad points are eliminated during this period, resulting in the model not obtaining a perfect model for a period of time. In the future, when dealing with missing points, it can be considered using fuzzy c-means clustering and other methods to complete the data to improve the prediction performance of the model.

Evaluation index of predictive performance

(1) MAPE

MAPE refers to mean absolute percentage error, which is expressed by the formula Eq. (10): (10)${\displaystyle MAPE=\frac{100\text{\%}}{n}\sum _{i=1}^n\left|\frac{{\stackrel{\wedge }{y}}_i-y_i}{y_i}\right|}$

where, y_i is true value of equipment temperature, and ${\stackrel{\wedge }{y}}_i$ is the predicted value of equipment temperature. The range of MAPE belongs to $\left(0,+\infty \right)$, MAPE value of 0% means perfect model, and MAPE value greater than 100% indicates relatively poor model. (2) RMSE

RMSE refers to root mean square error, which is expressed by the formula Eq. (11): (11)${\displaystyle RMSE=\sqrt{\frac{1}{\mathrm{n}}\sum _{i=1}^n{\left(\stackrel{\wedge }{y_i}-y_i\right)}^2}}$

where, y_i and ${\stackrel{\wedge }{y}}_i$ means the same with the formula (10); the range of RMSE is $\left(0,+\infty \right)$, and the error is positively correlated with RMSE value. When the predicted value is exactly the same as the actual value, it is equal to 0.

Comparative experiments

Aiming at verifying the effectiveness of this method, comparative experiments from two aspects are carried out in this article:

(1) The prediction performance under different characteristics is compared. Comparative features include only time feature T, only ambient feature A, only spatial feature S, multivariate information fusion feature vector MIFFV andthe reduced feature vector RFV, and CNN-GRU network is adopt as the prediction model to predict the temperature of phase A contact. The comparison results are listed in Table 2.

(2) The prediction performance of different models is compared under the same conditions. The reduced feature vector RFV proposed in this article is taken as the input data, and CNN-GRU network is compared with four other network models of BPNN (back propagation neural network), WaveNN (wavelet neural network, in which the Morlet wavelet is adopt), LSTM (long short term networks) and CNN-LSTM. During the comparative experiments, the parameters such as iteration times, learning rate and error threshold are the same in all prediction models. Prediction results of different models are compared in Table 3.

Analysis of prediction results

According to the above comparative experiments, this article analyzes the prediction results from multiple angles and draws the following conclusions from the statistical results from Tables 2 and 3:

1. CNN-GRU was applied to the prediction performance comparison experiment under different feature conditions, and the results showed that the multi-source information fusion feature vector constructed from the three aspects of ambient, time and space is better than the single feature prediction effect, in which MAPE and RMSE were reduced by one order of magnitude; that is, MIFFV includes rich information than the A, T and S feature;

2. The reduced feature vector RFV composed of principal components extracted after PCA dimensionality reduction had better prediction performance than MIFFV (MAPE is decreased from 6.98 to 5.48, and RMSE is decreased from 122.12 to 95.54), which shows that feature extraction plays a significant role in the prediction process, and the feature engineering scheme proposed in this article has the best effect on the temperature prediction of substation equipment.

3. Compared with CNN-LSTM, CNN-GRU had better performance, which shows that although GRU with two gating structures are simpler than LSTM three gating structures, GRU has better effect in temperature prediction of substation equipment;

4. CNN-LSTM had better effect than LSTM, which shows that CNN can mine the characteristics of equipment temperature depth when it is used for high-dimensional feature extraction, and provides a guarantee for the prediction model to achieve better prediction effect;

5. The depth network models of LSTM, CNN-LSTM and CNN-GRU had better prediction effect than the shallow networks of BPNN and WaveNN shallow networks, which shows that the deep learning network has obvious advantages in the field of prediction compared with the shallow networks in traditional machine learning.