Assessment of inter-rater and intra-rater reliability of the Luna EMG robot as a tool for assessing upper limb proprioception in patients with stroke—a prospective observational study

Type of study

The research was conducted in the form of a prospective observational study.

It was carried out in accordance with the ethical rules of the Declaration of Helsinki, and approved by the Local Bioethics Commission of the University of Rzeszow (resolution no. 2022/036/W). Written consent was obtained from all the participants. The study was registered in the clinical trials register at ClinicalTrials.gov (registration number NCT05486052).

Participants

The minimum sample size was determined prior to the study, using a sample size calculator (“PLUS module” from Statistica 13.3 software). The procedure identified a population size of 77 patients with chronic stroke. Ultimately, 126 patients with chronic stroke were enrolled for the study.

The study was conducted in a rehabilitation and spa therapy hospital (physiotherapy laboratory) in the Podkarpackie Region, Poland. The inclusion criteria were defined as follows: completed first ischemic stroke; patient’s informed, voluntary consent to participate; elementary (basic) gripping ability; the upper limb and hand paresis rated 4–5 on the Brunnström scale; degree of disability Rankin score of three; spastic tension of the upper limb and hand paresis not more than three on the modified Ashworth scale; and current health condition, confirmed by a medical examination, allowing participation in tests and exercises. The following exclusion criteria were applied: lack of the patient’s informed, voluntary consent; second or subsequent ischemic stroke of the brain; haemorrhagic stroke; stroke of the brainstem and cerebellum; cerebellum disorders of the higher mental functions limiting the ability to understand and perform tasks during examinations; visual disturbances; mechanical and thermal injuries potentially impairing the hand grip function; concomitant neurological, rheumatological and orthopaedic diseases, including neuropathies that can affect proprioception, permanent contractures that may affect the grasping ability and locomotion; unstable medical condition (i.e., cardiopulmonary disturbances, consciousness disturbances; convulsions, sudden acute chest pain, cardiac arrhythmias, dyspnoea); metal implants; electronic implants; menstruation in women and epilepsy.

The following inclusion and exclusion criteria were used based on available data in the literature (Rinderknecht et al., 2018; Kenzie et al., 2024; Tulimieri & Semrau, 2024; Findlater et al., 2018; Morreale et al., 2016; Cherpin et al., 2019; Andersen et al., 2009).

There were 200 patients with stroke in the rehabilitation and spa therapy hospital at the time of the study. Based on the inclusion and exclusion criteria, 147 patients were initially qualified to participate; however, after medical examination eight individuals were excluded. Ultimately 130 patients took part in Examination I, and 126 patients participated in Examination II (Fig. 1).

The study group comprised 126 individuals, including 78 women (61.9%) and 48 men (38.1%). The youngest participant was aged 45 (Min. 45.0), and the oldest 75 years (Max. 75.0), the mean age being nearly 60 years ( $\overline{x}$ = 59.9). The respective mean values of body weight (in kilograms), body height (in centimetres) and body mass index (BMI) were 68.3 kg ( $\overline{x}$ = 68.3), 170 cm ( $\overline{x}$ = 170.0) and 23.6 kg/m² (Table 1).

Procedure

The examinations were carried out using the diagnostic and rehabilitation device Luna EMG (EGZOTech Sp.z oo Gliwice, Poland). The assessments were carried out by two independent raters, under the same conditions and during the same time period. Two examinations of proprioception in relation to upper limb function were performed two weeks apart. The independent raters were physiotherapists with more than 5 years of experience working with people after stroke, trained in assessment, who had participated in previous published studies related to evaluation of the reliability of the Luna EMG Rehabilitation Robot to assess proprioception in the upper limbs in 102 healthy adults (Leszczak et al., 2024).

Luna EMG is a diagnostic and rehabilitation robotic device and its operation is based on reactive electromyography designed to train the sensorimotor cortex. The bioelectrical signals (EMG) obtained from the patient’s muscles show that the movement is active (Rinderknecht et al., 2018; Oleksy et al., 2022; EGZOTECH, 2019).

Calibration of the Luna EMG device is performed annually by a trained service technician. This process includes a thorough validation of the measurement functions to ensure the accuracy and reliability of the device’s readings. Regular calibration is essential for maintaining the device’s performance and ensuring the consistency of test results over time. Before the start of the examination, the Luna EMG measured the weight of the limb to unweight it during the test procedure, thereby eliminating its impact on the patient’s results. During the measurements, the patients remained in a sitting position, with the upper limb from the shoulder joint aligned with the trunk, and extended in the elbow, hips flexed at 90°. The trunk and the assessed limb were stabilised using straps. Each measurement comprised four tests performed in succession in each upper extremity. The range of motion was set between 0 and 90 degrees of elbow extension, which is limited by the device. The initial elbow joint position was 0° and the target position of elbow flexion was 60°; the movement was performed either actively or passively (Leszczak et al., 2024) (Fig. 2).

During the passive movement phase, the speed of the device was set to 30 degrees per second for the learning process and 10 degrees per second for evaluating the target position. The maximal torque for the device was maintained at 10 Nm, and the holding time during the learning phase of the target position (elbow flexion 60°) was set to 5 s.

Initially, the device passively conducted the patient’s movement, then the patient was asked to “remember the movement/setting” during which time the equipment stopped for 5 s and the patient was tasked with learning to sense their position. Then, after 5 s, the limb passively returned to its initial position (elbow extension 0°). The initial position was held for 3 s. After learning a passive movement, the patient was asked to actively perform the same movement with a stop: on the request, “please move your elbow to the position you learned and hold your arm in that position.” All tests were performed with the patient’s eyes closed. The duration of the study of one patient by two researchers was 30 min.

Statistical analysis

The analysis was carried out in the Statistica 13.3 program and the JASP program 0.18.3.0 (JASP Team (2024), Computer Software, Netherlands).

The mean and standard deviation values were first calculated for each series of measurements, and then for the differences between the series of measurements compared in the study. The significance of differences in the average level of the two measurement series was assessed using the t-test for dependent samples in which no significant values should be observed, but it should be noted that this is not a key factor in assessing the agreement of measurements. A comparative analysis of the measurement series was performed using Pearson’s linear correlation coefficient, as well as the key measure of agreement between two measurements, i.e., intraclass correlation coefficient (ICC). Bland-Altman method was applied as an alternative measure of agreement.

The value of p < 0.05 was assumed to reflect statistical significance.

Agreement between the measurements of right hand proprioception

As regards the assessment of the significance of differences, it was observed that Rater I on average recorded slightly higher measurement values in Exam I compared to those recorded in Exam II (p = 0.0001). A similar relationship was found between the measurements performed by the Raters in Exam I; on average the values recorded by Rater I were significantly higher compared to those identified by Rater II (p = 0.0220).

The above observations were not confirmed by Pearson’s linear correlation coefficients, which in all the analyses assessing both inter-rater and intra-rater agreement were very high (r = 1.00). The high agreement of the measurements carried out was also confirmed by intraclass correlation coefficients (ICC) (0.996–0.998) (Table 2).

The above findings are also confirmed by Bland-Altman plots. The highest mean deviation between measurements was identified in the assessment of agreement between Exam I and Exam II performed by Rater I (0.09°), and in a considerable majority of the results the differences were up to 0.4–0.5° (Fig. 3).

Agreement between the measurements of left hand proprioception

In the assessments of left hand proprioception the findings show no significant differences between the mean results recorded by the two Raters (p = 0.4487; p = 0.4027); however, the mean results recorded by each Rater in the two Exams differed significantly (p = 0.0002; p = 0.0001). Higher values were recorded by both Rater I and Rater II in Exam I. Nevertheless, ultimately the agreement between the measurements was confirmed by high Pearson’s linear correlation coefficients (r = 1.00) and ICC (0.994–0.999) (Table 3).

By interpreting the results shown by the Bland-Altman plots for left hand proprioception measurement it was possible to determine that the highest level of agreement existed between Rater I and Rater II in Exam II. Mean deviation was only 0.013° whereas deviations in the specific values did not exceed +0.4°. The highest mean deviations were observed in measurements recorded by Rater II during Exam I and Exam II (0.13°), with differences of nearly 2.5° in isolated cases. Despite the above observation, on the whole it can be said there is agreement between the measurements performed since a large majority of the differences in the results did not exceed 1.0° (Fig. 4).

Limitations

The study has some limitations, most importantly, evaluation of upper limb proprioception performed with the Luna EMG diagnostic and rehabilitation device was not supported with results acquired using proprioception assessment scales and tests. However, we are planning further research in which effects of stroke rehabilitation will be evaluated using Luna EMG as well as assessment scales and tests.