Using machine learning to analyze mental health in distance education during the COVID-19 pandemic: an opinion study from university students in Mexico

Background

The COVID-19 pandemic, declared by the World Health Organization (WHO) in March 2020, has had a profound and transformative impact on many sectors, especially in education (Karakose, 2021). The need to mitigate the spread of the virus led to the closure of educational institutions and the massive adoption of distance education modalities (Bozkurt & Sharma, 2020; Odeh & Keshta, 2022; Mahasneh et al., 2021). Globally, students have faced unprecedented challenges due to the abrupt shift to online education. These challenges include not only adapting to new forms of learning but also managing the uncertainty and stress associated with the pandemic (Marinoni, Van’t Land & Jensen, 2020). In particular, the psychological impact has been considerable, increasing levels of stress, anxiety, and other mental health issues among students (Qiu et al., 2020).

In Mexico, the impact has been even more pronounced due to the digital divide. Many students lack access to basic technological tools, which has exacerbated existing inequalities and limited their ability to effectively participate in distance education (INEGI, 2020). Distance education, though essential during the pandemic, has presented significant challenges in terms of student stress. Social isolation, the lack of face-to-face interaction with teachers and peers, and uncertainty about the future academic and professional have contributed to increased stress among students (Aucejo et al., 2020).

Machine learning (ML) has emerged as a crucial tool for analyzing and responding to the psychological impacts of the pandemic on students. Machine learning algorithms are being used to detect patterns of stress and negative feelings from data collected on educational platforms and online surveys. Studies such as those by Rahman et al. (2022) and Samsari, Mohamad & Selamat (2022) highlight the use of various machine learning techniques to predict and classify symptoms of stress and depression in students, demonstrating the capability of these methods to provide meaningful and timely analysis that supports psychoeducational interventions.

Factor analysis, a statistical technique used to identify latent variables underlying observed phenomena, has been applied to better understand how various factors contribute to the stress and feelings of students during the pandemic. This allows for a more accurate assessment of students’ needs and facilitates the development of more targeted and effective interventions (Costello & Osborne, 2019).

Introduction

In December 2019 the first cases of a new type of pneumonia in Wuhan, Hubei province, China appeared. The World Health Organization (WHO) described the infection due to the new coronavirus SARS-CoV-2 which produces the COVID-19 disease. Its global expansion has meant a public health crisis with an unprecedented crisis in modern times in accordance with WHO. At the end of January 2020, there were more than 7,000 infections all around the world; the consternation, uncertainty, and different feelings and thoughts took over the population (WHO, 2020). In March of that year, WHO (2020) had declared COVID-19 as a pandemic, which caused changes in everyday life; each person’s way of life experienced a 360 degree turn, and from then on nothing would be the same. This change affected the whole world in different levels; social problems started becoming worse, and deaths were reported daily around the world.

In Mexico, the first COVID-19 case was detected on February 27th, 2020. On April 30th, 64 days later, the number of patients increased exponentially, reaching a total of 19,224 confirmed cases and 1,859 deaths (Suárez et al., 2020). In the face of the world sanitary crisis, the Mexican government felt obligated to give a stay-at-home order to the population. For many people, quarantine has been a chaotic experience that changed their everyday activities completely. Moreover, as time passes, days of lockdown have been prolonged, which causes life to be paralyzed within the interiors of each home.

Means of communication inform that a growing number of people feel alone and scared, children suffer because of lockdown, domestic violence is increasing, and many people have lost their jobs and their financial means, and they live in uncertainty. Feelings of fear, anxiety, apathy, indifference, selfishness, desperation, and depression have appeared, because society hasn’t been able to adapt to the new way of life yet. Health, safety, economy, and education haven been altered. The latter has been affected as there was an extreme change in the way in which education processes were carried out. Approximately, more than 850 millions of children and youngsters remain far from schools and universities, with effective national closedown in 102 countries and local closedowns in other 11 (UNESCO, 2020). Under these circumstances, teachers and students face new challenges of adaptation, such as working from home and using technological tools within their reach.

In COVID-19 times it has been detected that some students do not dedicate enough time to their studies since they are immersed in different situations that take a toll on them; for example, not being able to understanding the academic topics makes them feel frustration, and even apathy towards dropping out. Apart from this, performance is affected due to the different geographical regions in which students are, because not all of them have the necessary means to take classes, and there are reasons to confirm that the access to technology is not enough (Krumsvik, 2020; Cervantes Holguín, 2020; Johnson, Saletti-Cuesta & Tumas, 2020). In the case of Mexico specifically, in 2019 just 44.6% of homes had a computer as a tool for school support; and 56.4% had internet connection (INEGI, 2020). Therefore, this research is designed to answer the following questions:

• RQ1. Is there a relationship between the feelings generated by lock-down and students’ behaviours in e-learning platforms required for assessment of mental health?

• RQ2. What are the attributes and machine learning algorithms required for detect stress and feelings effective?

RQ1 will be addressed in factor analysis of the virtual questionnaire applied to students and that allows defining the correlations between items. The identified factors will be used to a new construct for assessment of mental health, and RQ2 will be answered through the implementation of different machine learning algorithms.

In addition, the contributions of this article are:

• A scale to analyze the students’ mental health of the Instituto Tecnológico Superior de Misantla during the COVID-19 pandemic in Mexico.

• Machine learning models to detect university students’ the stress and feelings of the Instituto Tecnológico Superior de Misantla in distance education.

Related works

The hasty adoption of large-scale teaching, defined as emergency remote learning (Qiu et al., 2020; Priatna et al., 2020), has enabled some contradictions in the educational systems to come up (Stewart, 2021; Tulaskar & Turunen, 2022). The most evident are in the denominated high complexity schools in environments of high-risk social exclusion, where it has been confirmed that the social gap hinders digitalization of teaching (Álvarez et al., 2020; Rodríguez, 2020), even questioning the right to education (Román et al., 2020). The most recurring problems in long distance education include homes without WiFi, families with only one mobile device, inadequacy and lack of a living space, a poor cultural background of parents in the formal education field, and communication difficulties with professors and students. In many of these cases, the course will be lost, and there is even the risk of a long-lasting disconnection, maybe definite, with school (Vidal et al., 2020; Wahyuni, Wulandari & Hardhienata, 2021; Duman, 2023).

Attitudes towards information and communication technologies (ICT) are somewhat uncertain, the development of positive skills and attitudes to work with ICT in both academics and students demands a series of appropriate conditions for its development (Kalogiannakis, 2007; Nag & Roul, 2023; Vlachopoulos, 2013).

At the end of February 2020, when the global alarm because of SARS CoV-2 (COVID-19) began with more intensity, the World Bank established a group of global and multi-sector work to support countries to take necessary measures to face this threat. On April 20, 2020, school close-down were announced, impacting more than 91.3% of the student world population; a total of 1.6 billion, according to UNESCO (2020). As a result of the COVID-19 pandemic, the lives of children, parents and teachers have been altered. Although strategies have been put into practice which allow reducing the impact in education, the great inequality of opportunities that exists amplifies the problem.

In this sense, UNESCO (2020) is supporting the application of long-distance learning on a large scale, and it is recommending open educational apps and platforms that schools and teachers can use to reach a larger number of students. Moreover, practices to use mobile low-cost technologies for learning and teaching purposes are being shared to mitigate the distress that education is experimenting in underdeveloped countries.

The virus outbreak and the lockdown could be used as the best way to prove the effectiveness of interventions of educational technology for long distance learning. Unfortunately, few systems are completely prepared. China is one of the countries in which education continued, independently from school’s closedown, through internet and distance learning (Li et al., 2023). In less prepared countries, the access to technology in most homes can vary, and the access to quality internet or to smartphones is related to the socioeconomic class, even in countries with median income. Therefore, educational politics for the access to distance education must quickly focus on the most needed. Proper planning in the educational systems is something that was learned from the COVID-19 lockdown (Almusharraf & Khahro, 2020; Long & Khoi, 2020; Vachkova et al., 2022). This plan must include an introduction of protocols for evaluations in schools, launch campaigns of hygiene practices, imposition of school close downs, and the distance learning offer. In this sense, the educational measures during a crisis can help to prevent and recover public health, at the same time that they reduce the impact in students and their learning. Moreover, it is important to point out that education has the potential to contribute to children and youth protection (Bozkurt & Sharma, 2020; Wang et al., 2020).

In regard to higher education, the United Nations Educational, Scientific and Cultural Organization (UNESCO) has planned a series of recommendations in which it is mentioned that it is urgent that governments and institutions of those countries where the epidemic is starting to manifest, and that they plan proper measures to protect the health of citizens avoiding the violation of the right to education (UNESCO, 2020). Moreover, the educational authorities must supervise that these measures are put into practice to favor the fact that students continue to learn in spite of the temporary closedown of the institutions. In this sense, UNESCO (2020) recommends higher education institutions to take the following measures:

• Spread among the university community information and recommendations only and exclusively from sanitary national authorities and the WHO to avoid alarmism and the spread of rumors or fake news (UNESCO, 2020).

• Regularly use their own website and social networks so that the university community is informed on time and truthfully about COVID-19, including recommendations on behaviors to follow preventive or in case of infection, as well as the last advances in investigation counteracting in an active way racist or discriminatory attitudes and behaviors that could emerge as a result of distortions (UNESCO, 2020).

• Attend to the instructions and recommendations from national authorities and actively participate in the mechanisms of inter-university coordination to unfold in a coordinate and coherent way at national scale (UNESCO, 2020).

• The use of the university’s own platform for online learning, or the virtual campus, to continue to facilitate distance learning to students. In the case that the university does not have one, install one of the many open educational apps and platforms, taking into account that some of the students might only be able to use mobile devices (UNESCO, 2020).

This will demand having, in turn, mechanisms of training and support online for professors and students that should be correctly reinforced. This information is being updated and can be verified through COVID-19 infection maps (Urzúa et al., 2020; Adhikari et al., 2020).

In addition, the use of learning management systems (LMS), has grown considerably in universities around the world, which leads students to use their mobile devices; however, students find this platform as a repository for electronic documents and not as a learning tool (Papadakis et al., 2018).

Materials and Methods

In this research a survey was used to measure the feelings that students presented when using technological tools during the COVID-19 pandemic. The survey provides a quantitative understanding of the perceptions of university students in Mexico in terms of education and its relationship with the world pandemic. In addition, a factor analysis has been performed to measure the reliability of the construct and different machine learning algorithms were applied to detect university students’ feelings in distance education. Exploratory faction analysis (EFA) begins with the assumption that any observed variable can be linearly related to some underlying factors. The process involves extracting factors based on their eigenvalues, deciding the number of factors to retain through criteria such as Kaiser’s criterion or the scree test, and then rotating the factors to achieve a simpler, more interpretable structure (Steiner & Grieder, 2020). Various rotation techniques, such as Varimax (orthogonal) and Promax (oblique), can be applied depending on whether the factors are assumed to be correlated or not (Costello & Osborne, 2019).

Figure 1 shows the research methodology used which consists of three main phases that are described below.

Results

Visualization

The analysis of the first six questions showed uncertainty in students when receiving online education, due to a higher frequency in the answer “sometimes” in relation to “never” and “always”. Frustration (>50%) and stress (>50%) are the symptoms that are regularly present in students in relation to the results observed in Fig. 2. Moreover, students indicated a lack of concentration to carry out the academic activities at home during the period of contingency due to the COVID-19 pandemic. In this sense, the students indicated that sometimes (69%) or always (24%) felt too much workload in their academic tasks.

Dimensions

An exploratory graph analysis (EGA) was conducted for the variables defined in EFA and assess the number of dimensions; the dimensions in the graph are considered latent variables or factors. Figure 3 is a network diagram with two clusters, and red lines representing negative correlations and green lines representing positive correlations between variables, with the thickness of the lines indicating the strength of the correlation.

As expected, the first cluster (dimension) consists of variables Q2, Q3, and Q4, while the second cluster consists of items Q4, Q5, and Q8. Analyzing the graph, strong positive relationships (green) between Q4 and Q5 are observed, as well as negative relationships (red) between Q2 and Q4.

In addition, CFA was conducted to test the model proposed by the EGA network and results allowed for the relationship between the observed variables, Q2, Q3, Q6, Q7, Q4, Q5, Q8, and latent variables, factor 1 and factor 2.

The obtained value ${\chi }^2=11.412$ with a $p-value=0.576$. Since the $p-value>0.05$, there is not enough evidence to reject the null hypothesis, indicating that the model fits the data well. $RMSEA=0.000$ with a 90% confidence interval ranging from 0.000 to 0.045 and an associated $p-value=0.970$ for the $RMSEA<=0.05$ test. Therefore, this value indicates a good fit in the model.

Factor loadings for Factor 1 (see Table 5) and Factor 2 (see Table 6) are all statistically significant, indicating that all observed variables contribute significantly to their respective factors. For example, Q3 has a factor loading of 1.295 on F1, suggesting a strong positive relationship with that latent variable. Besides, the covariance estimate between factor 1 and factor 2 is −0.084 with a $p-value=0.000$, indicating a significant negative relationship between the two factors. However, the magnitude of this relationship is small. The negative relationship between the two factors suggests that academic load and pandemic stress decreases as online learning experience increases.

Table 5:

Statistical values of Factor 1 obtained by CFA.

Item	Estimate	Std-error	$\mathit{z}-value$	$\mathit{p}-value$
Q2	1.000
Q3	1.295	0.150	8.652	<0.001
Q6	1.098	0.142	7.749	<0.001
Q7	1.070	0.170	6.284	<0.001

DOI: 10.7717/peerj-cs.2241/table-5

Table 6:

Statistical values of Factor 2 obtained by CFA.

Item	Estimate	Std-error	$\mathit{z}-value$	$\mathit{p}-value$
Q4	1.000
Q5	0.882	0.111	7.981	<0.001
Q8	1.316	0.169	7.793	<0.001

DOI: 10.7717/peerj-cs.2241/table-6

Furthermore, Tables 5 and 6 show $p-values<0.001$, which reflects robust statistical significance in the associations between the observed variables and the latent factors. This uniform pattern indicates that the variables are not only significant predictors of the latent factors but also that the probability of these associations being due to chance is extremely low, reinforcing the robustness of our confirmatory factor model.

Figure 4 shows the resulting path diagram from the CFA, where the nodes labeled as F1 and F2 represent the latent factors. Each factor is associated with certain observed variables (Q2, Q3, Q6, Q7 for F1 and Q4, Q5, Q8 for F2), which are represented by rectangles. The solid arrows between the factors and the variables indicate the factor loadings, with the numbers above the arrows representing the standardized coefficients of the factor loadings.

Discussion

The findings of this research underscore significant contributions to the field of distance learning and student mental health in the context of the COVID-19 pandemic. Qiu et al. (2020), Priatna et al. (2020), Rodríguez (2020) have addressed the rapid adoption of distance education, highlighting the challenges and contradictions in educational systems, particularly in contexts of high social exclusion. These studies agree that the social divide negatively impacts the digitalization of teaching, a critical point also identified by Vidal et al. (2020), who emphasizes the risk of a prolonged and possibly definitive disconnection from the educational system for some students.

The current research complements these findings by investigating the psychological impact of confinement on students, using machine learning algorithms to classify students’ feelings about online learning. This innovative approach not only corroborates previous studies’ observations of the academic and emotional challenges faced by students but also offers specific technological solutions for their identification and management. The results show that stress and anxiety are prevalent among students due to confinement, and there is a clear relationship between these feelings and academic performance on e-learning platforms, aligning with concerns raised by Álvarez et al. (2020), Bozkurt & Sharma (2020) regarding the effects of the pandemic on education and mental health.

The use of ML techniques during the COVID-19 pandemic has proven to be indispensable not only for addressing educational and mental health challenges but also for informing policy and strategic decisions aimed at improving educational systems, especially in developing countries. The integration of ML allows not only the detection of patterns of stress and negative feelings in students, as observed in studies by Rahman et al. (2022), Samsari, Mohamad & Selamat (2022), but also provides guidelines for precise and evidence-based interventions.

We hope that these findings will be utilized by the scientific community and policymakers to develop more resilient and adaptive educational policies that can respond to emerging needs during similar crises in the future. In particular, ML models can help quickly identify areas of greatest need and distribute resources more effectively, which is crucial for developing countries with limited resources. Furthermore, previous studies have shown similar correlations between student stress and contributing factors during the pandemic, as indicated by research from Qiu et al. (2020), Aucejo et al. (2020), underscoring the consistency and relevance of our current findings. These studies not only validate our conclusions but also reinforce the need for a data-driven approach to understanding and mitigating the psychological and educational impacts of the pandemic.

Other efforts to use ML in similar contexts include work by Aggarwal, Girdhar & Alpana (2022) and Naiem et al. (2022), who have applied predictive models and sentiment analysis to assess the effectiveness of e-learning tools and to detect psychological support needs among students. These applications demonstrate the versatility and potential of ML to be adapted to various educational needs and contexts, offering a robust framework for future research and practical applications. With the application of these advanced techniques, we can not only better understand the challenges faced by students but also better anticipate and prepare educational institutions to face future crises, ensuring continuous and quality education despite adversities.

Conclusions

The present study set out to explore the relationship between confinement and changes in students’ psychological well-being, using e-learning platforms as tools to assess stress and negative feelings. The research focused on two main research questions: the first sought to establish whether there is a direct correlation between the feelings generated by confinement and behaviors observed on online learning platforms; the second focused on determining the most effective machine learning algorithms and attributes to identify signs of stress and emotional distress.

To address these questions, the study used a combination of quantitative and qualitative analysis, including monitoring students’ interaction with e-learning platforms and the application of advanced machine learning models to analyze data on students’ behavior and feelings. The findings confirmed that confinement has significantly exacerbated stress levels among students, affecting their academic performance and emotional well-being. The results presented here are based solely on self-report data. As such, they are inherently limited by the characteristics of this type of data, including self-selection bias and the potential for respondents to describe their experiences and feelings inaccurately or incompletely.

Furthermore, several machine learning algorithms were identified as promising tools for effectively detecting stress and negative feelings, thus offering avenues for early and personalized interventions. While the machine learning models demonstrated strong performance without requiring adjustments to additional parameters, it is important to acknowledge that the results might be influenced by factors not considered in this study. Future research should investigate additional variables that could impact the relationship between isolation and stress, including social support, personal resilience, or specific conditions relevant to the context of Mexican universities.

This study achieves its overall objective by demonstrating through empirical evidence how confinement has impacted students’ mental health. Using a rigorous methodological approach along with machine learning technologies, new possibilities are opened for monitoring and intervening in mental health within educational environments. The relevance of adopting adaptive and responsive strategies to promote student well-being in crisis situations is emphasized. Consequently, the conclusions of this analysis suggest the need to focus more on the psychological facets associated with virtual learning, proposing to effectively integrate educational technological resources with emotional support, marking a path forward to improve support for students.

Supplemental Information