A hybrid integration framework based on LOOCV and SARIMA: relationship exploring and predictive analysis between discipline attention and literature research

Materials

The research object is from China National Knowledge Infrastructure (CNKI). Specifically, all the literature-themed “Junior High School Information Technology” from 2011 to 2023 was searched in Chinese Journal Full-text Database (CJFD), and keywords were extracted using keyword frequency analysis. This article first calculated the keyword frequency for each year using the complete counting method of word frequency analysis, then accumulated the word frequencies of the same keywords for all years and output them in reverse order. Finally, all keywords in the information technology subject of junior high school in the past decade were counted. These keywords were used as preliminary pre-selected objects of junior high school information technology subjects and further filtered as required.

The quantity of academic articles is indexed by the quantity of CNKI literature of the keywords collected. The subject network attention is measured by Baidu Index. In this article, network attention refers to the degree to which netizens pay attention to a certain social hot topic or thing in the era of the internet. When this kind of attention is specifically applied to a discipline, it forms the discipline network attention. The discipline network attention reflects the degree of people’s interest in research hotspots, article topics, or content related to the discipline. Usually, it is measured by the number of searches for relevant information about the subject, such as keywords, academic articles, etc. This indicator reflects the popularity and influence of the discipline on the Internet, and the Baidu Index is an important tool to measure this indicator. The Baidu Index is based on the record of massive search data of netizens on Baidu’s search platform, which can reflect social hot spots, and netizens’ interests and needs over a period of time. Specifically, Baidu Index is derived by weighting the search scale and frequency of keywords based on Baidu’s massive search behavior data of netizens. It provides a quantitative indicator of netizens’ attention and has been widely used by many scholars in various fields in recent years (Chen et al., 2023; Liu et al., 2024), with high authority and universality. Therefore, this article chooses Baidu index as the quantitative index of subject network attention.

This article collects the relevant data from the CNKI and Baidu Index, and analyzes the relationship between discipline literature researches and discipline network concern.

RAPA framework

According to the two dimensions of disciplinary attention and literature research, this article constructs a hybrid integration framework for relationship analysis and predictive analysis (RAPA) by using multiple methods comprehensively (Fig. 1). Meanwhile, this article verifies the effectiveness of the RAPA framework from the perspective of information technology discipline.

Firstly, this article takes the discipline of information technology as an example and extracts the main keywords of the discipline by word frequency analysis. Then, based on the main keywords, it divides two dimensions: disciplinary attention and disciplinary literature research.

Secondly, according to the two dimensions divided, this study collects their respective datasets as the basic data source for the RAPA framework.

Finally, this article explores the specific application process of the RAPA framework using several methods such as Pearson Correlation, LOOCV, SARIMA, and Predictive regression, further verifying the effectiveness of the RAPA framework implementation.

The specific implementation process of the RAPA framework includes three steps, which correspond to three research questions RQ1, RQ2, and RQ3, respectively. Each step used a unique method for experimental data analysis, which is described in detail below.

Step 1: For RQ1, RAPA analyzed the relationship between subject network attention and subject literature researches.

Selecting representative data of subject network attention (Baidu Index) and CNKI subject literature researches, RAPA used the Pearson correlation coefficient to measure the relationship between subject attention and the number of its literature researches. The Pearson correlation coefficient can reflect the linear correlation between two variables and has been used in multiple studies (Alhawarat, Abdeljaber & Hilal, 2021). Therefore, this article referred to the Rahadian et al.’s (2023) practice and used quantitative data to analyze the correlation between them.

Step 2: For RQ2, RAPA predicted the future trend of subject network attention and subject literature researches.

Based on the above relationship, RAPA first established a regression model between subject attention and subject literature quantity by cross-validation method; secondly, using time series to predict subject attention; finally, the subject attention and the regression model were used to predict the subject literature quantity. The time series prediction method based on network attention has been applied to many fields (Wang et al., 2023; Yadav & Thakkar, 2024). This study drew on Luo et al.’s (2023) approach and predicted the future development trend of information technology in junior high school.

Step 3: For RQ3, RAPA explored the differences between different predictive methods.

Based on the prediction results of the subject literature researches, RAPA compared the prediction accuracy and differences between the method in this article and gray prediction (Aykaç Özen & Öbekcan, 2023) using two evaluation indicators, Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE). The comparison was used to verify the validity of the prediction method and the reliability of the experimental result in this article. The following was a detailed introduction to specific research methods.

Relationship analysis and modeling

Based on RAPA framework, this article used correlation analysis, cross-validation and other methods to explore the relationship between subject network attention and subject literature researches.

Predictive analysis

Based on RAPA framework, this article made a prediction analysis on the basis of the relationship between subject network attention and the number of academic papers. First, RAPA used the SARIMA model to predict the network attention of the keywords in the next 2 years based on historical observations. Then, according to the regression relationship model between the subject’s attention and its literature research, the publication number of the literature researches in this subject is further predicted.

Construction of predictive models

Based on the information technology subject of junior high school, this article predicts the subject attention according to the selected keywords. Considering the limited space, this article takes the keyword “Flipped classroom” as an example to analyze the construction process of the prediction model in detail. Other keywords are similar.

Sequence analysis

Figure 2 showed the original sequence of the ‘flipped classroom’, which exhibited a trend of regular fluctuations. By dividing the original time series into three parts: Trend, Seasonal, and Residual term, to observe the seasonal characteristics of the data. Figure 2 showed that the time series has a relatively obvious seasonal trend, with annual data showing a trend of first increasing and then decreasing. Meanwhile, the ADF test results of the original sequence were obtained (Table 3). Table 3 showed that p = 0.944730, much larger than 0.05, with unit roots. Therefore, the original sequence was non-stationary and required differential processing to meet the stationarity characteristics.

Table 3:

ADF test results of the model.

Test statistic	p-value	Critical value (1%)	Critical value (5%)	Critical value (10%)
−0.145348	0.944730	−3.510712	−2.896616	−2.585482

DOI: 10.7717/peerj-cs.2754/table-3

Sequence stabilization and testing

From a statistical point of view, only stationary time series can avoid the existence of “pseudo-regression”. This article first performed first-order difference on the original sequence, followed by seasonal difference, and drew a test chart for the differential sequence (Fig. 3). Combining Fig. 3 and the ADF test, it was found that the mean and variance of the sequence after differential processing have stabilized. The p = 7.852714e−11, less than 0.05, indicating that the sequence is stable after one ordinary difference and one seasonal difference.

Model parameter determination

Model parameters can be determined by observing the trailing and truncating phenomenon of Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF) graphs. However, this method may not be entirely accurate and is subject to a certain degree of subjectivity. Therefore, it is necessary to combine multiple fitting models and select the appropriate model based on the AIC, BIC, and residual analysis results. Based on the maximum likelihood estimation method, this article used the auto_arima function in the pmdarima library for parameter estimation. Finally, the optimal model was selected according to AIC and BIC. The smaller the AIC and BIC, the better the fitting and prediction performance of the model.

In this case, ARIMA(1,1,1)(2,1,0)12 was selected as the optimal model according to the results of model operation and considering the significance of estimated parameters. By substituting the trained optimal model into Formula (3), the basic expression of the flipped classroom model was obtained:

(8) $$(1-{\phi }_{1}L)(1-{\alpha }_1{L}^{12}-{\alpha }_2{L}^{24})\mathrm{\Delta }{\mathrm{\Delta }}_{12}{y}_t=(1+{\theta }_{1}L){ϵ}_t.$$

By inputting the ARIMA(1,1,1)(2,1,0)12 model result parameters into Formula (8), the detailed expression of the model can be obtained as follows:

(9) $$(1-0.9237L)(1+0.702L^{12}+0.5288L^{24})\mathrm{\Delta }{\mathrm{\Delta }}_{12}{y}_t=(1-0.5844L){ϵ}_t$$

Model evaluation

Based on the estimated parameters, this article selected December 2019 to December 2023 for model evaluation and drew a model evaluation diagram (Fig. 4). According to Fig. 4, the trend of the fitted value and the actual value was basically consistent. The model has a good prediction effect. Therefore, this model can be used for subsequent prediction.

Similarly, build the model of Information-based teaching, Micro-lesson, Multimedia teaching and Educational informatization, and predict the future attention trend according to the modeling results.

Prediction of subject attention

Referring to the modeling and prediction process of the keyword “flipped classroom”, this article predicts other keywords to reflect the attention trend of junior high school information technology subjects (Fig. 5).

The prediction results of subject attention represented by keywords show that the hotspot of information technology subject will show a flat or declining trend in the next 2 years. Among them, information-based teaching, flipped classroom, and multimedia teaching show a stable trend, which is the same as the level of concern in recent years. However, Micro-lesson and educational informatization are showing a downward trend. Especially in the field of educational informatization, by 2025, its attention index will drop to around 175. The hotspot of information technology subject presents an unstable trend, and there has been a state differentiation. As a new type of teaching resources and methods, Multimedia teaching and flipped classrooms play an important role in integrating information technology and classroom teaching. It is also a hot topic that various disciplines will continue to pay attention to in the future (Qin & Lu, 2018). Since the rise of educational informatization, it has comprehensively promoted the use of modern information technology in the process of education and teaching. However, comparative analysis shows that China should continue to promote the construction of digital education infrastructure and improve the efficiency of digital education resources (Wang & Chen, 2022).

Prediction of the number of subject literature researches

The number of subject literature researches is obtained by substituting the predicted values of disciplinary attention into the regression equation. First, according to the relationship model between subject attention and literature research (Formula 7), the independent variable in 2024 was substituted into Formula (7) to obtain the dependent variable value. By further performing anti-logarithm treatment on the dependent variable, the number of literature researches on junior high school information technology in 2024 could be obtained, which was 145.

Secondly, to reduce prediction error, data from 2024 was added to the sample set and a regression prediction model with LOOCV was constructed. The resulting model is as follows:

(10) $$lny=-0.753+0.361x_1+0.286x_2-0.108x_3+0.701x_4-0.109x_5.$$

Then, the independent variable of subject attention in 2025 was substituted into Formula (10). Further anti-logarithm treatment could obtain the number of literature researches in 2025, which was 123. Figure 6 shows the publication number of literature research in this subject over the next 2 years. It can be seen that the overall number of publications in junior high school information technology literature has shown a fluctuating trend over the years. After 2023, the publication number of academic literature tends to be flat. The number of publications has decreased compared with previous years, with an average of 136 articles.

An overall analysis of RQ2 found that the discipline attention represented by the five keywords presented a steady or declining trend. In recent years, the public’s attention to information technology subject in junior high school is not high, showing a slowing trend. Meanwhile, its academic literature researches would not increase in the next 2 years. Information technology subject is a compulsory course in primary and secondary schools, which is of significance to the reform of classroom teaching methods. Properly increasing literature researches of this discipline can effectively tap the key points within the discipline and promote the in-depth integration of multiple disciplines.

Prediction comparison

Based on the relationship between subject attention and literature research established in RAPA framework, this article predicted the future publication number of information technology subject themes in junior high school. To evaluate the prediction effect and ensure the necessity of establishing the above relational model, RAPA also used GM(1,1) (Althobaiti & Shabri, 2022; Yeh & Chang, 2021) to forecast the number of articles in this discipline. The two prediction methods were compared and analyzed to solve RQ3. In addition, RAPA selected the Standard Deviation (SD), MAE and MAPE of the prediction results as evaluation indicators for prediction accuracy. Table 4 listed the index values of GM(1,1) and attention-based regression prediction methods.

According to the overall prediction accuracy in Table 4, the multiple regression prediction error combined with disciplinary attention was relatively small. It indicated that the method of using subject attention to predict literature quantity was of high accuracy. It also showed that network concern can assist discipline literature research and improve the accuracy of prediction. Specifically, the standard deviation of multiple regression prediction results was small, indicating that the forecast value was nearly the actual value, and the prediction result was relatively well. According to the calculated prediction results, there was little difference between the data results of the two methods, which also confirmed the correlation between the subject network attention reflected by the Baidu index and the amount of CNKI academic articles. Meanwhile, this also showed that it was reasonable to use the attention to predict the subject literature researches.

The implications of the findings

This study aimed to explore the relationship between subject network attention and literature research from the perspective of information technology in junior high school. The main research results were summarized in three aspects, namely the correlation between disciplinary attention and literature research, the prediction results of disciplinary attention and its literature research, and the comparison of prediction accuracy. The results indicated a certain dynamic correlation between subject attention represented by the network index and the amount of literature research in scientific journals. The public’s behavioral willingness to pay attention to disciplines is recorded through big data, while their contribution to scientific research is reflected in the output of scientific journal literature. In recent years, the amount of information technology subject literature in junior high school has not increased significantly, which may be related to the public’s attention for the future development of this discipline. The results of each problem were discussed in detail in this article.

Keyword extraction and relevance criterion analysis based on the RAPA framework found that the Trending Keywords of information technology discipline in junior high school were Information-based teaching (r = 0.887**), Micro-lesson (r = 0.841**), Flipped classroom (r = 0.798**), Multimedia teaching (r = 0.867**), and Educational informatization (r = 0.753**). It represented the research focus of information technology curriculum in Chinese middle schools, and also reflected the diversification of teaching methods. This was similar to the views of Huang & Wang (2020), who stated that diversified teaching methods of information technology in primary and secondary schools infiltrated and crossed each other, gradually moving towards integration. This was a realistic choice to improve and innovate teaching methods. In addition, the correlation calculation results showed that the r-value between the subject network attention represented by multiple keywords and the number of academic articles was greater than 0.75, showing a positive correlation. This indicated that online media and the scientific journals had a correlation in disseminating scientific knowledge, but the timeliness of their communication content was unknown. While scholars have conducted comparative studies on this timeliness and found that scientific journals can detect hot issues in the professional field earlier than online media, and guide the public to pay attention to these issues (Li, Feng & Tian, 2017). Currently, online media and scientific journals are in parallel, which not only satisfy the public’s pursuit for distinctive content, but also drive the development of the subject.

Time series prediction based on the RAPA framework found that the attention of information technology subject in junior high school will show a flat or declining trend in the next 2 years. Moreover, the academic hot topics represented by keywords present a trend of differentiation. Information-based teaching, Multimedia teaching, and Flipped classroom are strategic ways to integrate classroom technology, improve interactivity, and enhance learning experience (Gao et al., 2020). The trend of its future attention remains the same as in recent years, indicating that the public still holds expectations for the application mode of integrating technology in future middle school information technology classroom. The attention to micro-lessons and educational informatization is showing a downward trend, especially educational informatization, whose attention index will drop to 175. Upon investigation, it was found that the future implementation of educational informatization still faces some challenges, such as issues with equipment, training, and resources. These challenges indirectly affect the attention index. In addition, regression prediction found that the number of literature in this discipline will decrease in the next few years after 2023, with an average of about 136 articles. This is consistent with Zhao et al.’s (2023) prediction trend on the literature researches of high school information technology subjects. She suggested that one of the reasons for the low number of scientific journal’s articles may be the homogenization of many literature contents. Therefore, the hot topic attention and literature researches of this discipline need to be improved.

A comparative analysis of predictions based on the RAPA framework found that the error in predicting literature researches using disciplinary network attention was relatively small. Specifically, compared with other methods such as GM(1,1), MAE in this study was reduced by 3.51 and MAPE by 1.14%. This indicates that disciplinary network attention, as an auxiliary variable in literature research, contributes to the rationality of quantitative analysis in disciplinary literature research. This was similar to the research idea of Duan & Pan (2017) and Pai, Li & Liu (2019) using real-time data of network media as intermediary variables to identify emerging disciplines. In the big data environment, network media data has strong dynamism, and the data characteristics of different disciplines are different. Therefore, the attention reflected by online media data helps researchers in different disciplines to grasp the frontier and hot spots of the discipline, and promote the progress of the discipline and science.

Impact and recommendations of the findings

The result of this study is very enlightening to the information technology subject development of in secondary school. The information technology field has always been in rapid development and transformation, and secondary school information technology subject needs to keep up with the latest technological trends in a timely manner. The results suggested that the public’s attention to the future of this discipline was not high, and the number of its literature researches was at a medium level. Information technology subject in secondary schools is a subject to train students’ practical operation and problem-solving abilities, and also an important way to cultivate their quality. Therefore, it is necessary to strengthen the development path of this discipline, such as focusing on innovation and creativity, integrating interdisciplinary knowledge, emphasizing information literacy, etc., to enhance the public’s attention to this discipline. At the same time, attention should be paid to the phenomenon of de-homogenization in literature research in this discipline, expanding the multi-dimensional research content. This is conducive to the long-term development of information technology subject in secondary schools.

In addition, the results also showed the correlation between the subject network attention and its literature researches. This reflected from the side the dynamic correlation between online platforms and scientific journals. The era of online big data has brought unprecedented challenges to traditional scientific and technological journals, changing their dissemination methods and achieving digital transformation of journals. However, their timeliness is different. The emergence of journal research hotspots often occurs earlier than the peak of online attention (Li, Feng & Tian, 2017), which helps the public better grasp the direction of online public opinion. At the same time, network attention can also help scientific journals to capture social hot issues, promoting the transformation of traditional scientific journals in the information wave. Therefore, in the process of teaching research, it is necessary to comprehensively utilize network big data and traditional data to fully reflect the more comprehensive data value, thereby improving the quality of literature research.

Research limitations and future work

This study also has some limitations. First, it only explored the correlation between discipline concern and the quantity of the academic article in the view of middle school information technology subjects. The extrapolation of research conclusions is limited. The applicability of other disciplines needs further investigation and discussion. Second, the attention data selected in this study comes from the Baidu index. Although its search engine has a great influence, there may still be statistical bias. Attention data reflects the behavior dynamics of a large number of Internet users, which may be reflected on different platforms. This article did not collect and verify data of other platforms.

Given these limitations, more comprehensive and extensive sample-size studies will be necessary in the future to validate and extend the current findings. First, the generalizability of this study’s results to other disciplines is unknown. Therefore, future studies can be conducted from disciplinary perspectives, such as English and mathematics, to explore the relationship between the attention of different disciplines and their literature researches. While expanding research findings, summarize the characteristics of different disciplines. Secondly, future research can further expand the sample types based on the data sources. For example, Altmetrics Attention Score (AAS) is a weighted metric containing online real-time data of 17 different types of network media on Altmetric.com’s comprehensive platform (Altmetric, 2024). It can measure the social and academic influence of literature research in a diversified and timely manner, making the evaluation more comprehensive, specific, and objective. In addition, 360 Trend is also a commonly used search engine (Trend, 2024). Therefore, future research can combine different indices to measure attention to compare the similarities and differences between different Index platforms.