A novel virtual reality application for autonomous assessment of cervical range of motion: development and reliability study

Participants

Participants were recruited among students and teachers from the Universidad San Pablo-CEU. The inclusion criteria included men and women of all races between 18 and 65 years of age without cervical joint pain in the last month and no prior treatment for neck pain. The exclusion criteria were a previous medical diagnosis of: visual impairment that was not corrected with the use of glasses or contact lenses; migraine headache; complex regional syndrome; previous surgeries in the neck and head region; sensory or vestibular alterations; existence of tumors in the craniocervical region; previous fracture in the head or neck region; osseous malformations in the thoracic, cervical, or cranial regions; or idiopathic or otogenic vertigo/dizziness. Following the recommendations used to conduct a reliability study (Koo & Li, 2016), 30 subjects were recruited.

VR setup

VR is defined as “the use of computer-generated virtual environments and the associated hardware to provide the user with the illusion of physical presence within that environment” (Jayaram, Connacher & Lyons, 1997). VR setups usually are comprised of an HMD and controllers that allow the user to interact with the virtual environment. HMDs are head-worn devices that project images into the user’s field of view while allowing free mobility of the head and, potentially, the entire body (Rahman et al., 2020; Lee, Chang & Park, 2020). Most HMDs that are currently on the market have displays with high-definition resolution, a large field of view, and high refresh rates. Some VR setups have the option to track additional parts of the body, not just the head (tracked by the HMD) and the hands (tracked by the controllers). These VR setups include trackers that can be attached to the part of the user’s body where movement needs to be followed. An example of such a commercial tracker is the HTC Vive Tracker™, which can be attached to objects such as sport equipment, toy weapons, or even body parts (Niehorster, Li & Lappe, 2017).

Since the introduction of VR in the world of computing, it has been used for different purposes, including gaming, education, industrial applications, and medical applications. In the field of medicine, VR has been used for training medical procedures (Tin, Hertelendy & Ciottone, 2021; Moon et al., 2021; Nambi et al., 2021), to evaluate sight (Yasuda et al., 2020; Sabu et al., 2020), and in rehabilitation (Chen, 2021; Laver et al., 2015; Tin, Hertelendy & Ciottone, 2021).

VR provides a virtual environment that immerses its users. Presence is the sensation that the user feels when truly immersed in a virtual world (Rubin, 2018), i.e., the subjective part of the experience. Immersion is the objective part of the experience, where the visualization of the images, frames projected per second, audio stimuli, and kinesthetic stimuli (Găină et al., 2021) are taken into consideration. VR tracks the users’ neck movements with high accuracy to provide better immersion. Therefore, we hypothesize that we can take advantage of this precise tracking to measure CROM. Additionally, through the usage of tracker devices, the movement of any other body part, such as the trunk, can also be tracked. This could permit the identification of compensatory movements.

Our measuring instrument was a VR setup that was comprised of one HTC Vive Pro Eye™ HMD, one Vive Tracker™, two wireless HTC Vive Controllers™, and two Lighthouse™ v2 base stations. The HMD has a resolution of 1,440 × 1,600 pixels per eye, a refresh rate of 90 Hz, a field of view of 110°, and built-in Hi-Res Certified headphones.

The HMD uses room-scale tracking technology to interact with the virtual environment using Valve’s SteamVR™ Lighthouse system. The Lighthouse base stations contain infrared LEDs and a laser array that sweeps in the horizontal and vertical directions. The surface of the HTC Vive™ devices (the HMD, controllers, and trackers) has photodiodes that measure when the laser has hit them. Knowing the time differences at which the different photodiodes are hit by the laser enables the accurate tracking of the three-dimensional position and orientation of the devices. One of the advantages of this tracking system is that it provides six degrees of freedom that track both the translation and rotation of the user within the virtual environment. To calculate the CROM using an HMD, we used the rotation axes. The movement in the sagittal plane corresponds to flexion-extension (X-axis), the movement in the transverse plane corresponds to right-left rotation (Y-axis), and the movement in the frontal plane corresponds to right-left lateral flexion (Z-axis) (see Fig. 1).

A tracker was used to follow the movement of the subjects’ backs in order to measure trunk compensations. It was attached to the back of the subjects over the fourth thoracic spine (T4), since this sensor placement has shown good reliability in previous research (Raya et al., 2018; Tsang, Szeto & Lee, 2014) and may prevent collisions with the HMD during cervical extension movement. When placing the tracker on the subject’s back, they were asked to stand in a T-pose position. The T4 spinous process was identified by palpating the T3 spinous process (aligned with the spine of the scapula) (Han et al., 2012) and moving one segment immediately inferior. The tracker has a blue triangle drawn on its center, and one of the sides of the triangle was placed over T4. This tracker position also ensures that the tracker has a large contact surface with the back, minimizing any movement or slippage (see Fig. 2A). To attach the tracker, we used a custom-built X-shaped harness formed by an elastic band and a 3D printed adapter (see Fig. 2). The elastic band is 2 m long and 20 mm wide, with a zigzag one mm flexible silicon stripes to prevent slippage (see Figs. 2B–2C). The silicone stripes were on the side of the strap that was in contact with the subject to prevent the displacement of the tracker. The harness consists of two handles placed in such a way that each of them was on one arm, at shoulder height.

The handheld controllers were used to track the position of the hands and control the application using the trackpad and trigger buttons. The tracking of the hand’s position was used to increase the sensation of immersion, since in the virtual environment, the avatar representing the user has hands that replicate the hands’ position in the real world. Although all devices were tracked, only the data from the HMD and the tracker (to identify potential compensatory movements) were stored for CROM assessment. The stored data contained the three-dimensional position and rotation of both devices sampled at a fixed rate of 90 Hz, which is the maximum fixed-interval rate provided by the VR setup. The game engine updated and recorded the timestamped position and orientation in a CSV file to be processed later. Since OpenVR was used, this sampling rate had to match the framerate of the VR device used (HTC Vive Pro Eye™).

To guarantee that the measurements were completely blind between raters, it was decided that each rater use an independent computer. The computer used by rater A was a HP OMEN Intel Core i7-10750H with 16GB of RAM and an 8GB NVIDIA GeForce RTX 2070 Super GPU, running the 64-bit version of Windows 10 Pro, SteamVR™ 1.16.10, and the CROM assessment application described in the next section. The computer used by rater B was a DELL G5 5590 Intel Core i7- 9750H with 16GB of RAM and a 6GB NVIDIA GeForce GTX 1660 Ti GPU, running the 64-bit version of Windows 10 Home, SteamVR™ 1.17.16, and the CROM assessment application. Both computers were powerful enough to run the VR experience at least at 90 frames per second, with a refresh rate that was supported by the HMD without missing frames.

VR application

We developed an application to guide the subjects while they performed the movements necessary for CROM measurement. The VR application was developed using the Unity 3D game engine (v.2019.4.19f1) with OpenVR and the Steam framework for HTC Vive.

The VR application for CROM assessment was made up of two interfaces or viewpoints: the VR experience that is seen by the user through the HMD, and an interface shown in the monitor of the computer that only the rater observes (see Fig. 3A). The rater interface has a menu in which a number uniquely associated with the subject that is being evaluated is entered; this number is used to identify the evaluation sessions that correspond to that particular subject. The rater interface also has control buttons that allow for starting and stopping the assessment and exiting the application.

The virtual environment is a medical/physiotherapeutic office (see Figs. 3A–3E). It is sparse in order to avoid distractions during the movements. One of the more important considerations when designing the room was to avoid clear lines that crossed the whole ceiling since they could serve as a guide for the subjects during movements and influence the trajectory of the movements or their range.

The virtual room contained a desk, two chairs with a human figure sitting in one of them, another chair where the avatar of the subject is sitting, and some other medical furniture placed further away. The figure is placed in front of the subject (see Figs. 3B–3E). This figure performs, as a way of example, the movements that the subject will need to carry out. Audio instructions played by headphones built in the HMD also guide the user through the process and specify the movements that need to be performed. At the beginning of these instructions, arrows visually indicate the direction of the movements (see Figs. 3B–3D). When performing the movements, a large black number in the center of the field of view indicates what repetition the subject is in (see Fig. 3E). Using the inverse kinematic solver in Unity, the head and hands of the virtual avatar are synchronized with the person wearing the HMD and the controllers. Thus, the user can view the virtual body from the first-person perspective, enhancing the sense of embodiment or body ownership.

As shown in the application flowchart (see Fig. 4), during the first measurement, the VR experience reproduces an audio clip with a brief description of the objective of the evaluation, as well as detailed indications of the movements that the subject has to perform. Then the experience guides the subject through warm-up exercises similar to the movements that are performed for CROM assessment, which are also exemplified by the human figure. In addition to the warm-up series, the subject must perform three repetitions of each movement. Each repetition is indicated by a large number that appears in the center of the screen (see Fig. 3E). At the end of the third repetition of a given movement, the explanation of the next movement begins. Once the three movements have been completed, a message informing the user that the evaluation session has ended is played. After the first measurement, the VR experience only provides short instructions about the movements, since the complete detailed instructions were already presented during the first measurement. Since the subject has already performed one warm-up series and, at least, a complete series of movements for CROM assessment, the warm-up series is not performed during the subsequent measurements. Similar to the first measurement, each repetition of each movement is indicated with a large number that appears in the center of the screen and is illustrated by the figure. At the end of the third repetition of each movement, the explanation of the next movement begins.

The tracking data from the HMD and tracker have a sampling rate of 90 Hz. They were stored in a CSV file as follows: sample number, timestamp, device ID, X, Y, Z, qW, qX, qY, qZ, aX, aY, and aZ. The timestamp follows the HH: mm:ss.ffff format. The device is identified with a single letter: H for the HMD and T for the tracker. X, Y, and Z correspond to the device position in the space; qW, qX, qY, and qZ to the quaternion; and aX (Pitch), aY (Yaw), and aZ (Roll) are the Euler angles (see Fig. 1) (Kong, 2014). To enable the reproducibility of the results of this study, as well as to enable other studies to analyze the data collected, a transcript of the audio message played in the VR application in both its long and short versions, and the 120 CSV files corresponding to each of the four sessions of the 30 subjects are provided as Supplemental Material (see File S1).

Experimental protocol

Before starting the measurements, the physical area was prepared. A 2 × 2 meter room area was physically delineated and framed on the floor using masking tape before starting the assessment of any subject. In this way, the virtual room used in all of the sessions corresponded to the same physical space. As shown in Fig. 5, the base stations were placed at opposite corners of this area on top of tripods with their positions also marked on the floor. The chair in which the subject was seated was placed at the center of this area and its position was also marked on the floor to ensure that the same location was used for all the subjects. The computers of rater A and rater B were in front of the subject chair to facilitate changing the computers between measurements.

In SteamVR™, the room setup is the procedure in which the limits of the physical area where the interactions in the virtual environment can take place are established. This configuration only needs to be done once, as long as the position and orientation of the base stations are not changed. If the base stations are moved or their orientation is changed, the room setup must be carried out again. The space used for the measurements was also used for other purposes and it was necessary to remove the base stations at the conclusion of each measurement session. Hence, to ensure that the measurements were taken within the same area every time, a room setup procedure was performed at the beginning of each measurement session for each of the two computers. Between three and five individuals were assessed during each session. During the room setup, the tape marks of the delineated floor area were used as guidelines to generate a 2 × 2 meter virtual room. If the mapped area was not equal to 2 × 2 m, it was manually edited in SteamVR to obtain the desired size. In all cases, the virtual room was oriented in such a way that the virtual chair faced the computers, since this was also the position of the real chair in which the subject was seated.

Two evaluators (rater A and rater B) supported the subjects when using the VR equipment and controlled the CROM assessment application from their respective computers. Both raters are engineers with experience using and developing VR applications and are familiar with the CROM assessment application. The rater who carried out the first assessment for each subject was selected randomly based on a sequence of random numbers generated with a computer. While an assessment was being performed by one rater, the other waited outside the room.

The measurement protocol of this study was made up of four CROM assessments for each subject, alternating between raters. Before starting the first assessment, the randomly chosen rater explained to the subject the operation of the VR application and the correct use of the controllers. No instructions about how to perform the movements for CROM assessment were given by the raters at any point; all these instructions were provided by the VR experience. The raters limited their explanations and their support to the usage of the VR equipment and the placement of the tracker on the back of the subject. In accordance with the current COVID-19 health protocols, during the experiments the subjects wore face masks and were also provided with disposable eye masks to prevent their facial skin from coming into direct contact with the HMD.

The subjects could use either hand to control the application with the VR controllers. By pressing any trackpad, the subject was placed in the VR experience behind the figure that illustrated each movement to be performed. The trigger button was used to start the measurements. The order of the movements that each subject must carry out followed a random sequence generated automatically by the computer. Each sequence contained the movements of flexion-extension, rotation, and lateral flexion in a random order without any of them being repeated within the same sequence.

The first rater started the VR application for CROM assessment. The display on the computer screen indicated that the first evaluation is going to be carried out, and a warm-up and detailed instructions on how to perform the movements was given. During the assessment, the raters oversaw the subject without intervening. At the end of the measurement, the rater removed the HMD and tracker from the subject. The first rater then left the room and the other rater entered the room to start the second evaluation. During the second assessment, there was no warm-up stage and the VR experience provided only abbreviated instructions on how to perform the movements. The third and fourth assessments were carried out in a similar way to the second, including the removal of the HMD and tracker between them.

CROM calculation

To perform the CROM calculation, we used Matlab R2020a. Each subject’s data were processed individually. The CROM assessment VR application stored the HMD and tracker’s pitch, yaw, and roll, which corresponded to flexion-extension, rotation, and lateral flexion, respectively. The angles recorded from the VR setup were in the range (0–360). Using a wrapping function, we mapped them to the interval (−180–180). In this new interval, the 0-degree position corresponded to the head of the subject when in a neutral position before starting the movement where the helmet was perfectly aligned with the subject’s head. Under this assumption, in the flexion-extension movement, the positive angles corresponded to the flexion and the negative to the extension. For rotations, the positive angles corresponded to the right rotation and the negative to the left rotation. For the lateral flexions, the positive angles corresponded to the left lateral flexion and the negative to the right lateral flexion. In practice, there will be some misalignment between a vector normal to the subject’s head and a vector normal to the helmet, which will result in a small (positive or negative) angle in the neutral position, which needs to be corrected.

To calculate the CROM for each movement, we started by plotting the corresponding angle recorded by the HMD versus time. No filtering or any other type of signal conditioning was applied to the time series since it was not deemed necessary. Visually, the positions where the three repetitions of the movement were identified, and we manually selected a time position slightly before the first repetition in which the subject’s head was in a neutral position. This position was approximate and small variations in its identification should not affect the final result of the measurement. From this selection, the average value of the first 10 measurements was calculated; this value corresponded to the estimate of the misalignment between the helmet and the subject’s head in a neutral position. This average was subtracted from each angle measurement to correct the misalignment. Next, we manually selected a time position slightly before the start of each repetition, and another time position slightly after the end of each repetition (see Fig. 6).

The maximum and minimum peaks of each repetition were identified using the manually selected start and end times. The CROM value of each repetition was calculated as the difference between the maximum and the minimum of the movement. Following the same approach used in Romero Morales et al. (2017) and Raya et al. (2018), we calculated the average CROM value of the three repetitions, obtaining the final value for the flexion-extension, rotation, and lateral flexion when the angle being processed corresponded to pitch, yaw, and roll, respectively. These were the measurements without the correction of the compensatory movements made with the trunk.

The temporal positions that were manually selected for the calculation of the angle corresponding to the neutral position and the beginning and end of each one of the three movements, were stored. These same positions were used for the calculation of the CROM with the correction of the compensatory movements. To this end, the value of the angle recorded by the tracker at each instant was subtracted from the angle recorded by the HMD. The resulting angle corresponded to the movement made only with the head. The compensation-corrected CROM measurement was then calculated over the corrected angle time series using the temporal positions stored during the calculation of the CROM without this correction. The person in charge of the data processing described in this section was one of the engineers who also acted as a rater.

Effect of compensation correction on the measurement error

In this study, we hypothesized that the use of the tracker to correct trunk compensations should increase the reliability of the measurements. However, it is possible that the reliability added by the correction does not compensate for the measurement error accumulated using multiple instruments. Let us represent the measurements of the HMD and the tracker as:

(1)${\displaystyle RO{M}_{HMD}=CROM+Trunk\text{\_}compensation+Measurement\text{\_}error}$ (2)${\displaystyle RO{M}_T=Trun{k}_{compensation}+Measurement\text{\_}error}$

where ROM_HMD is the measurement of the HMD; ROM_T isthe measurement of the tracker; CROM is the movement performed with the neck; Trunk_compensation is the compensation movement made by the subject’s trunk; and Measurement_error corresponds to the HMD and tracker’s measurement errors which, for the sake of simplicity, are considered to be of equal magnitude for both devices. Within the measurement error, we included the error inherent to each measuring instrument, which in this case is given by the tracking accuracy of the VR setup (Passos & Jung, 2020), plus any possible user/procedure error. We believe that the instrument error is dominated by this second source and is mainly caused by the slippage of the HMD or the tracker relative to the head and back of the subjects, respectively. When the compensation correction is applied to the HMD measurements, the instrument errors are not canceled out, but instead increased due to error propagation: (3)${\displaystyle CRO{M}_C=RO{M}_{HMD}-RO{M}_T=CROM+2Measurement\text{\_}error}$

where CROM_C is the CROM measurement with compensation correction. Therefore, if Trunk_compensation is less than Measurement_error, the error of the CROM_C measurement is greater than the error of the measurement obtained directly from the HMD without applying any compensation correction (see Eqs. (1) and (3)). A greater reliability of the corrected measures with respect to the uncorrected ones would indicate that Measurement_error is less than Trunk_compensation, while a lower reliability would point towards the opposite.

Statistical analysis

The statistical analysis was performed using version 27 of the IBM SPSS Statistics statistical package. Since our sample size was less than 50, the Shapiro–Wilk test was used as a test of normality (Ahmad & Khan Sherwani, 2015). The level of statistical significance was established as p < 0.05. Subsequently, the mean and standard deviation (SD) were calculated to characterize the CROM obtained in each of the three movement axes for the different subjects.

The ICCs were calculated, and the ICC (3, k) model was used for intra-rater analysis, and the ICC (2, k) model was used for inter-rater analysis (Koo & Li, 2016). Reliability aims to measure the coherence between repeated measurements of the same variable in the same individual that is subject to the same conditions. While intra-rater reliability deals with the consistency of a single rater evaluating the same event on two different occasions, inter-rater reliability focuses on measuring the variability of two or more raters evaluating the same event on the same occasion (Koo & Li, 2016). ICC values from 0 to 0.5 are considered poor, between 0.5 and 0.75 considered moderate, between 0.75 and 0.9 considered good, and above 0.90 considered excellent (Koo & Li, 2016).

The standard error of measurement (SEM) and minimum detectable change (MDC) were calculated (Portney, 2020). The SEM can be estimated using the ICC: ${\displaystyle SEM=S_P\sqrt{1-ICC}}$

where S_P is the pooled standard deviation of measurements. MDC is useful to characterize the effectiveness of a device to detect changes in the measured variable and can be calculated using the SEM. When computed at the 90% confidence level (Kolber et al., 2011; Marszałek & Molik, 2019; Laird, Kent & Keating, 2016), MDC is given by: ${\displaystyle MD{C}_{90}=1.65xSEMx\sqrt{2}.}$

System usability

To measure the usability of the VR system as a CROM assessment tool, we used the Spanish version of the System Usability Scale (SUS) (Sevilla-Gonzalez et al., 2020). The SUS is a quick and easy-to-use tool for both system users and researchers. It provides a single score on a scale that is easily understandable (Bangor, Kortum & Miller, 2008). The SUS questionnaire consists of 10 questions designed to be answered by the user after interacting with the system. The questions alternate between positive and negative statements to avoid respondent bias. Each statement can be scored with a value between 1 and 5, where 5 is “Strongly agree” and 1 is “Strongly disagree”. The odd questions are positive statements, and they are scored by subtracting one point from the rating of the user. The even questions are negative statements, and they are scored by subtracting 5 from the rating of the user. When adding all of the scores, a scale of 0–40 is obtained which, when multiplied by 2.5, becomes a scale of 0–100 (Brooke, 1995). For a system to have good usability, it must obtain a result between 68 and 84 points. If the system obtains a score higher than 85 points, it is considered to have excellent usability (Sevilla-Gonzalez et al., 2020).

When all four CROM assessments were finished, the subjects were asked to anonymously fill out the SUS questionnaire to evaluate the VR experience.

Limitations

This is the first study to investigate a novel VR experience that could assist clinicians in the evaluation of people with neck pain or associated disorders. However, this study has several limitations. The results were limited to the characteristics of the study population, which included asymptomatic, mostly young adult Caucasian participants from a university community. Although this sample may be appropriate to evaluate the general usability and reliability of the VR system in the evaluation of CROM, future research should also investigate whether the results could be influenced by age, previous experience with digital technology, or the presence of craniocervical pain conditions.

A limitation that could affect the repeatability of the study is that during data processing, we made a manual selection of the beginning and end of each movement, although the subsequent calculation of the CROM across different planes from these positions was automatic. This could be improved by developing an algorithm that automatically detects the beginning and end of the movements.

Although the usability and reliability of the VR system ranged between good and excellent, future studies should also evaluate its validity compared to other established CROM measurement instruments.