Mobile phone enabled mental health monitoring to enhance diagnosis for severity assessment of behaviours: a review

Search strategy and information Source

The strategic purpose of digitised assessments for psychological health issues has been immediately required during the COVID-19 pandemic. Wearable gadgets can be used to enhance assessments of mental health issues and can be used for tracking at-risk and quarantined populations. Furthermore, while passive monitoring does not operate as frequently as active monitoring, it can collect and generate vast amounts of data, which may be used in supporting clinical assessment. As a result, the search focused on identifying the most appropriate keywords for collecting recent papers on the subject such as mobile technologies, ambient sensing, sensors and wearable and clinical competencies. Searches were conducted in electronic bibliographic databases such as PubMed (Health Science), IEEE Xplore, ACM Digital Libraries (computing methods), Soups (information article), APA PsycInfo, and Mendeley Data (Physio informatics), addressing this via query-based searches.

1. Scopus (ScienceDirect)((ALL (Monitoring deceives)) OR (ALL (Passive monitoring)) OR (ALL (hardware)) OR (ALL (Software)) OR (ALL (smartphone deceives)) OR (ALL (‘monitoring AND deceives’ OR ‘Passive AND monitoring’ OR ‘hardware AND software’ OR ‘smartphone AND deceives’))) AND (ALL (well-being * OR physical * OR mental * AND health))

2. IEEE Xplore digital library(“Full Text Only”: mental health monitoring) AND (“Full Text Only”: hardware) OR (“Full Text Only”: software) AND (“Full Text Only”: passive monitoring) AND (“All Metadata”: mental behaviours) AND (“All Metadata”: well being)

3. APA PsycInfo Any Field: mental health monitoring using hardware OR Any Field: software with passive sensing

4. Mendeley ALL(mental AND health AND well AND being AND using AND passive AND monitoring AND sortBy = publicationYear)

5. PubMed((“mental health”[MeSH Terms] OR (“mental”[All Fields] AND “health”[All Fields]) OR “mental health”[All Fields]) AND (“health”[MeSH Terms] OR “health”[All Fields] OR “well”[All Fields] OR “well being”[All Fields]) AND (“passive”[All Fields] OR “passively”[All Fields] OR “passives”[All Fields]) AND (“monitor s”[All Fields] OR “monitorable”[All Fields] OR “monitored”[All Fields] OR “monitoring”[All Fields] OR “monitoring s”[All Fields] OR “monitoring, physiologic”[MeSH Terms] OR (“monitoring”[All Fields] AND “physiologic”[All Fields]) OR “physiologic monitoring”[All Fields] OR “monitor”[All Fields] OR “monitorings”[All Fields] OR “monitorization”[All Fields] OR “monitorize”[All Fields] OR “monitorized”[All Fields] OR “monitors”[All Fields])) AND (2003:2022[pdat])

6. ACM [All: mental health behaviors prediction using passive monitoring] AND [All: mental health well being using passive monitoring]

Various reputable databases are excluded so the risk of duplication of research is minimized, for example, the Web of Science database.

Study selection

Studies were extracted based on the query formed above and refined based on their titles, followed by their abstracts. The rest of the refined articles were further scrutinised by reviewing the content and determining relevance to the research question. The search list was ordered chronologically and then analysed by a group of researchers in unison so as to minimise selection bias.

Figure 1 shows the data flow diagram explaining how the publications for the review were chosen based on elimination carried out manually with inclusion and exclusion criteria. The systematic search described above yielded a sample of 943 manuscripts.After removing the duplicates, we were left with 514 studies. 349 of the 514 titles examined were deemed eligible for abstract evaluation. An estimated 199 abstracts were considered as potential review candidates, which led to 182 full-text retrievals and evaluations. Finally, 115 manuscripts were included in the study. Table 1 shows the number of returned and selected papers from retrieved databases.

Based on the exclusion criteria Table 2 shows the count of papers that were excluded from the final stage of the review. Table 3 shows the number of articles published per year in the last six years.

Summary of the search results

Tables 4 and 5 provide the summary of various behavioural health outcomes from sensors, wearable remote monitoring intervention studies which include duration of the evaluation, target population, methods involved, resultant outcomes, clinical outcomes, hardware, software components used for monitoring.

From Figs. 2 and 3 and Tables 4 and 5 it can be seen the majority of review articles considered for analysis chose the accelerometer and GPS as a source of the user’s data can be seen. The accelerometer provides added benefits such as covered distance, speed, static/inactive and time periods of movements which also provided amiability, movement and confinement. Moreover, the GPS provides dynamic, location variance, entropy, circadian movement and universally common latitude and longitude data from the satellite to predict the intensity of movement. It can also predict sociability, and detachment that can provide context information to predict the mental health issues more accurately.

Analysis of the results

The framework for the health assessment system represents in Fig. 4 was created to guide the analysis. This framework includes the combination of embedded sensors values used to deduce the majority of behavioural classifiers (markers) to forecast the inferences such as mental health issues, academic performance, fall detection for elderly people, isolation from the social group and even for lifestyle recommendations. Those behavioural markers such as physical activities, usage of the phone, sleep, location and social activities are detected from the sensors data. Finally, inferences are compared with the self-reports as described in Table 6 (ASRM, SGABS, GABS, IBS, HADS, PHQ, GHQ) which are generally used by clinical experts to analyse the health issues. This section deals with a detailed review of the embedded sensors and their function, which can result in monitoring various health metrics like behaviour monitoring, physical activities, overall well being, sleep quality, student monitoring and conviviality. In addition, traditional methods used by practitioners were later combined into scales given as self-reports for mental health monitoring.

Embedded sensors in smartphones

A smartwatch resembles a wristwatch in appearance, but it can do much more than keep time. Bluetooth is supported on digital timepieces, and the functions can be extended to smartphones. In such instances, the smartwatch can be used to read messages, accept calls, monitor the climate, and many more advanced functions.

In addition to these advantages, smartwatches aid in the analysis of the wearer’s behaviour and the determination of their mental health. The researchers used a combination of sensors and wearable technology (Table 7). Smartphones (71/115, 61.7 percent), wristbands or smartwatches (44/115, 38.2 percent), were the most commonly utilised gadgets in the research of behaviour monitoring. Accelerometers (46/115, 40.0 percent), mobile phone usage (36/115, 31.3 percent), Global Positioning System (33/115, 28.6 percent), actigraph (23/115, 20 percent), microphones (28/115, 24.34 percent), electrocardiogram (ECG) , and Electroencephalogram EEG (were the most commonly utilized sensors (23/115, 20.0 percent).

From Table 8 it is clear that 72.1 percent of research papers (83/115) used this sensor to collect data from users, most of which were related to physical activity. In smartwatches, the accelerometer is used to detect the wearer’s movement and orientation. Regardless of whether one or two hands are used, the accelerometer in the smartwatch detects about two dozen movements and activities (Lindner et al., 1999). The controls for software applications are then mapped to these motions. The Tri-axial version is a common smartwatch modification that maintains track of the wearer’s physical activity. The Tri-axials record up and down, side-to-side, and back-and-forth movements, unlike the uniaxial version, which only records up and down movements (Carbonell, Michalski & Mitchell, 1983).

In order to enhance the prediction of behavioural characteristics, context-based scenarios have to be added by using location data from the GPS in devices. The user’s location and movements are tracked with this GPS data in most studies 37.39 percentage (43/115). It can be either used alone as an individual sensor or combined along with Bluetooth and Wi-Fi networks. The level of sociability among the users can be easily measured in addition apart from the location measured from the Bluetooth. To determine the wearer’s orientation and angular velocity, gyroscope sensors are used. Gyroscope sensors are more advanced than accelerometers in terms of functionality. These sensors are capable of tracking both lateral and tilt orientations. Accelerometers, on the other hand, can only track linear motions (MacInnes, 2003). A revolving disc called the motor is mounted on a spinning axis in the gyroscope’s design. With the help of Earth’s gravitational field, this sensor detects the wearer’s orientation (Liu et al., 2018).

Out of the refined papers, SMS and calls were used to analyse the depressed mood and social avoidance in (13/115) 11.3 percent of papers, gyroscope and microphone were used for passive sensing in seven percent (8/115) and 5.2 percent (6/115), respectively. The major work of a microphone is to communicate with the opposite parties. By analyzing this factor, the predictions that can be made are mood (modulation of your voice), drowsiness and isolation, while a gyroscope is used to measure the basic day to day activities.

Apart from the data received through sensors, some other additional information can also be collected by the utilization of user patterns for handling the mobile. The communication made using the phone such as calls, text messages, as well as usage of devices incidents such as screen ON/OFF, time spent on the phone, lighting and settings of the device, are used to detect health-related data. Application usage like ambient light and phone screen on/off status were considered as a key factor in 7.82 percentage (9/115) selected publications to predict the sleep of the users. Table 8 illustrates the rest fitness data, such as the camera, and magnetometer, which can be collected from embedded sensors and mobile phones.

The ability to collect data passively is one of the key benefits of using smartphones for health monitoring. All sensor data comes from the smartphone’s omnipresent sensors, and passive data collection means there is no user interaction or participation. To detect physical activities, 52 of the 115 publications employed data from only two or more sensors, the majority of which were accelerometers. Various articles additionally used GPS, camera, and speedometer on their own as described in 115 articles involving the usage of several sensors. In the majority, 23 articles predominantly use the accelerometer, and in conjunction with GPS, Wi-Fi, gyroscope sensors and microphone to determine the physical activities and overall general behaviour.

Another important issue that affects the accuracy of the collection of sensing data in the monitoring systems is the operating system. Building for Android and IOS, the two most popular phone operating systems, has its own set of issues and problems. Android is presently the most widely used operating system, with the benefit of being simple to write (Wang & Zhang, 2015). The quicker sensor scanning rates of this operating system have been demonstrated (Boonstra et al., 2017). Furthermore, IOS prohibits third-party apps from operating in the background indefinitely, preventing data collection (Hossain & Poellabauer, 2016). 73.6 percent (67/91) of the chosen papers developed their solution exclusively for Android devices, 18 for both Android and IOS, and six did not specify which operating system they used.

Many of the publications identified using the phones’ sensors because of efficiency in energy and reduced overhead in collecting data from users in the passive matter. Advancement in various platforms in machine learning fields provides various tools to develop prediction models to analyse and interpret data in studies using more than three sensors.

Statistical analysis

Physical activities and mental health, as well as sociability, academic achievement monitoring, and overall well-being, are the most examined health features among the publications chosen for this systematic review, as shown in Table 9.

Self reports-traditional scales used to analyze mental health

Before releasing smartphone-based passive monitoring systems to the general public, it is necessary to do extensive testing to guarantee that users are engaged with the device. These scales are used to evaluate and compare results like behaviours, length of study and number of participants involved to access the study. The data were collected using a cross-sectional survey method, which entails gathering data at a single moment in time (Lindner et al., 1999). During one phase of data collection, the phenomena under inquiry are captured as they present themselves. In order to cross-validate and test the recent instrument’s measurement, clinical assessment scales are used. Table 6 shows the major valuation scales used (Polit & Hungler, 1993; Lindner et al., 1999; MacInnes, 2003; Bernard, 1998). The data collected by smart mobile phones can be compared and validated using ground truth data. The sort of data utilised as base ground truth was disclosed in around 60.8 percent (70/115) of the evaluated papers, while the other research provided no relevant information. However, there are several drawbacks to this strategy, such as the fact that users may not always react precisely, resulting in inconsistent bipolar outcomes.

Challenges in monitoring systems

The created systems discussed in this scoping review still have several flaws that must be addressed in order to meet the expectations and demands of users.

• validation of monitoring systems

  • These systems should be tested with a population sample that is highly representative of the target population for a sufficient period of time in order to collect enough data and produce as accurate results as possible. From the survey, 71.1 percent (84/115) of the selected papers tested their system with up to 50 participants, and 17.7 percent (21/115) of the selected papers tested their system for one to three weeks, which appears to be a short period to ensure reasonable results and ensure users’ confidence in using available solutions.

  • Majority of the system designed so far are assessed on younger generations additionally outputs may not be much reliable (Cheffena, 2015; Del Rosario et al., 2014). Single population alone was targeted in the vast number of research works that may be overly customized, resulting in erroneous findings when the systems are utilised by other groups (Chen et al., 2014; Beck, 1979). Personalized models outperformed general models, according to other studies (Huang et al., 2015; Juen, Cheng & Schatz, 2015; Saeb et al., 2017a; DeMasi et al., 2017).

• Furthermore, 43.2 percent (51/115) of the papers chosen demand users to keep their smartphones close to them or to use them in a specified bodily position, such as their hand, chest, or trouser pocket. Other research required users to have their handsets near them (Bowling, 1991; Huang et al., 2015; Ellis et al., 2012) or to keep them turned on all the time to ensure that the system worked properly (Boonstra et al., 2017; Staples et al., 2017). Because smartphones become an invasive technology for users under certain situations, they may be rendered obsolete.

End users issues

• In terms of privacy, 26.08 percent (30/115) of respondents indicated that privacy concerns could cause consumers to abandon their commitment. All those data are related to their personal these have to be kept secure were concentrated in (Kelly, Curran & Caulfield, 2017; Wang et al., 2014). In some studies users are not willing to save their personal data on cloud environment (Ben-Zeev et al., 2017; Aslam et al., 2016) or in their internal mobile devices by hashing techniques (Huang et al., 2015; Boonstra et al., 2017; Aslam et al., 2016; Wang et al., 2014), and there are sensor which contains certain concerns about privacy (Juen, Cheng & Schatz, 2015).

• Still on the subject of technological considerations, consumers anticipate that the software will not take up too much space or memory and that it will be able to run in the background without interfering with other smartphone functions.

Behavioral marker issues

Seasonality and the environment are two other essential factors to consider. In the winter, for example, GPS and accelerometer data in Minneapolis will differ from that in Miami.

Variability in sensing data is due to the decentralized sources, including data types, people’s characteristics, and diverse situations. Sensors in cellphones differ from one manufacturer to the next, from one model to the next, and from one version to the next, impacting the data collected. The link between constructs or how people use measurement equipment may be influenced by their properties. For example, age may be linked to the number of social contacts, with older people having fewer contacts and desiring less contact, but it could also be linked to how social engagement is monitored using a phone (e.g., older people are more likely to call and less likely to send text messages than younger people).

Finding a scientifically valid balance between identifying uniform variables, to making data pooling simple (e.g., using the same questions) statistical methods is used to manage heterogeneity by providing similarity. It is challenging to pool data points if they are not identical. Since the field of personal sensing in mental health is not mature, some agreement may be possible on a core set of clinical assessment methods (EMA or self-report), which will allow for uniform anchoring of a wide range of sensor data as it evolves and changes over time and between research projects. A fully integrated assessment is complicated by the wide range of devices, sensors, and data permissions available.

Motion artefacts (MA) are common in EDA data gathered with wrist wearables. Variations in the pressure put on the EDA electrodes as a result of the wearable tightness, hand movements, or wrist rotation may cause severe data distortion. Many researchers have used approaches such as exponential smoothing, filtering, and adaptive denoising based on the wavelet transform to suppress artefacts in the past. MA suppression approaches, on the other hand, have the issue of indiscriminately filtering the entire time series data, resulting in distortions even in artifact-free areas. As a result, a new approach called MA detection was developed, which tries to efficiently encode expert knowledge on artefact recognition into a machine learning classifier model.

The technical novelty over generalisability research works appear to address the same behavioural marker often use different sensors, different sets of features, different methods of measuring behavioural markers, and different research designs (e.g., giving people phones versus having them use their own, studying them for varying periods of time, or having varying numbers of participants excluded). The machine learning methods utilised differ, and the results or weightings, especially for group models, are not always comparable between research. As a result, studies looking at the use of machine-learning approaches to estimate behavioural markers show that it is possible under certain conditions; nevertheless, the dependability required for clinical use has yet to be established.

Future work

Existing solutions on the other hand, have various flaws that must be solved in order to meet user expectations, such as privacy and battery issues. First, just a conceptual analysis was provided rather than a numerical analysis of the extent and type of the research. Second, a fully integrated assessment is complicated by the wide range of devices, sensors, and data permissions available. Thus, the sensor data from the monitoring system will be more accurate if a two-stage technique is used with an initial artefact detection phase followed by localised categorization depending on the target population. Finally, due to the limited sample size, different methodology, and varying research duration, we were unable to use a systematic quality grading system or draw conclusions using quantitative meta-analysis.