Novice assessors demonstrate good intra-rater agreement and reliability when determining pressure pain thresholds; a cross-sectional study

Design

Using a cross-sectional, observational test-retest study design, ten novice raters determined PPTs using two types of digital algometers to assess intra-rater reproducibility parameters. For agreement, the SEM was used as absolute value for the measurement error and the CV was used as relative value for the measurement error compared to the mean PPT of the muscle (Mokkink et al., 2010). For reliability, the ICC_1,1 was used as ratio between the measurement error (Mokkink et al., 2010). All intra-rater reproducibility parameters were measured per algometer.

Data were acquired between February and June 2019. The study was approved by the Medical Ethical Committee of the University Medical Center in Groningen, The Netherlands (METc 2016.613; M17.207169). All participants and raters signed an informed consent prior to the measurements. Reported reproducibility parameters are based upon the COSMIN group recommendations for studies on psychometrics (Gagnier et al., 2021; Mokkink et al., 2010; de Vet et al., 2016). The Guideline for Reporting Reliability and Agreement Studies (GRRAS) was used (Kottner et al., 2011; Gerke et al., 2018).

Raters

Novice raters were students in the last year of their undergraduate physiotherapy program. Ten raters (six females, four males) were recruited from the Hanze University of Applied Sciences in Groningen, The Netherlands. Physiotherapy students without experience in determining PPTs were eligible to participate. Exclusion criteria were being physically unable to perform PPTs on the selected muscles (in terms of positioning and strength) or not being able to communicate in the Dutch language. Their mean (SD) age, weight and height were 23.0 (3.0) years, 75 (11) kg and 179 (6) cm. For this project, all raters attended two four-hour training session to perform PPT measurements with a Somedic and a Wagner digital algometer. The training included an explanation of the study procedures, standardised participant instructions, palpating and marking of the body locations (approximately 2 h) and the use of the two digital algometers (for approximately 3 h per algometer). Practising assessment of PPTs was performed on healthy participants who did not participate in the main study.

Participants

Pain-free, healthy participants were recruited from the staff and student populations of the Hanze University of Applied Sciences, Academy of Health Care, in Groningen, The Netherlands, and the local community. The selection of healthy participants was based on the aim to determine rater-dependence in PPT measurements and as such to eliminate potential confounding variables of dysfunction or disease. Convenience sampling took place via an announcement on the university’s intranet and an email was sent to all staff and students. All participants underwent a short screening to determine eligibility (Gierthmühlen et al., 2015). Inclusion criteria were being pain-free, between 18 and 65 years old and an adequate understanding of the Dutch language. Participants were not permitted to use alcohol, caffeine or nicotine-containing products 2 h before testing, or using analgesics 24 h prior to testing (Girdler et al., 2005; Baratloo et al., 2016; Bagot et al., 2017). Exclusion criteria were the presence of acute or chronic pain, neurological, orthopedic or cognitive disorders, pregnancy or if participants had undergone surgery to the legs, arms or neck.

Sample size

Sample size estimation was performed for both agreement and reliability parameters for the group of raters. For agreement, using an upper bound of the 95% CI of +15% of the SEM, a minimum of 81 participants was required (per algometer) (Stratford & Goldsmith, 1997). For reliability, based on α = 0.05, β = 0.8, an expected ICC_1,1 of >0.7 with the lower bound of the 95% CI of 0.5, a minimum of 63 participants was required to calculate reliability per algometer (Walter, Eliasziw & Donner, 1998).

PPT measurements

After collecting the demographic, anthropometric, lifestyle and psychosocial data, participants were randomly allocated to a rater and went into one of four rooms where they received standardised instructions. Then, every participant was familiarised with the measurement on the dorsal aspect of the hand by the allocated rater. Figure 1 shows the procedure and methods used in the study. Four rooms were used to simultaneously measure all participants. The rooms were comparable regarding size, temperature and noise levels.

PPTs were measured using either a Somedic digital algometer (Type II; Somedic AB, Stockholm, Sweden) or Wagner digital algometer (Force Ten FDX 25; Wagner, Greenwich, CT, USA). Both digital algometers were used with a 1 cm² rubber tip with an application rate of ~50 kPa/s. The Somedic algometer provides visual feedback regarding the application rate and has a hand-held switch for the participant to press when the feeling of pressure changes into painful pressure. The Wagner device does not provide feedback regarding the application rate. Consequently, the raters had to estimate the rate themselves. Because visual feedback about the slope might assist raters in obtaining more stable PPTs, the two different algometers were used to assess whether the intra-rater reproducibility parameters are influenced by the algometer properties. Furthermore, the Wagner algometer has no patient switch, so participants verbally indicated when their pain threshold was reached.

After marking the locations with a skin pencil, raters measured the PPTs three times in each location, with an inter-stimulus interval of 20 s to avoid sensitisation of pain (Reezigt et al., 2021; Nussbaum & Downes, 1998). The measurements were taken on the dominant side, in a fixed order in two circuits. In the first circuit, with the participant in sitting, PPTs were determined on the tibialis anterior muscle (~5 cm distal to the tibial tuberosity) and rectus femoris muscle (~15 cm proximal to the patella base) on the dominant side, alternating between the two locations (Fig. 1B). In the second circuit, with the participant in prone and the dominant arm above the head, PPTs were determined between extensor carpi radialis brevis muscle (~5 cm distal to the lateral epicondyle of the humerus) and the paraspinal muscles at the C5-C6 level of the neck, again in an alternating manner (Fig. 1B). The test locations were selected based on the different levels of complexity of the measurements and height of the PPT. Furthermore, these muscles are often selected in PPT studies and their reproducibility parameters are available for experienced raters (Sterling et al., 2002; Ylinen et al., 2007; Jones, Kilgour & Comtois, 2007; Jørgensen et al., 2014; Waller et al., 2016; Jakorinne, Haanpää & Arokoski, 2018; Srimurugan Pratheep, Madeleine & Arendt-Nielsen, 2018; Knapstad et al., 2018; Middlebrook et al., 2020).

After removal of the skin markings and a 5-min wash-out period, the rater re-marked the locations and retested the PPTs.

Randomisation and Blinding

Participants were randomly allocated to either being measured (1) with the Somedic or Wagner algometer, and (2) by one of the ten raters. Computer-generated randomisation and opaque envelopes were used. To blind the raters for the measurement results, they handed the algometer to a research assistant after each measurement. The research assistant read the value on the display, recorded the result, reset the algometer and handed the device back to the rater. Participants were also blinded for their test values by shielding the display of the algometer during the measurements.

Statistical analysis

Group reproducibility

All within-session and between-session agreement and reliability values for the Somedic and Wagner algometer are presented in Table 2. Within-session agreement, as SEM, ranged from 42 to 72 kPa, and as CV from 10.8% to 14.5% for the Somedic algometer. For the Wagner algometer, it ranged from 42 to 74 kPa (SEM), and from 11.5% to 14.8% (CV). Measurements at the rectus femoris muscle showed the best within-session agreement, whereas the measurements at the extensor carpi radialis brevis muscle showed the lowest within-session agreement using either algometer (Fig. 2).

Between-session agreement ranged from 36 to 71 kPa (SEM) and from 10.4% to 16.1% (CV) for the Somedic algometer, and from 47 to 76 kPa (SEM) and from 9.4% to 14.6% (CV) for the Wagner algometer. The best between-session agreement was found at the rectus femoris muscle using either algometer (Fig. 2). Measurements at the extensor carpi radialis brevis muscle showed the lowest agreement for the Somedic algometer (CV: 16.1% (95% CI [14.3–18.4%]) on an average PPT of 393 kPa), whereas measurements at the paraspinal muscles at C5-C6 were the lowest for the Wagner algometer (CV: 14.6% (95% CI [13.0–16.7%]) on an average PPT of 325 kPa). The values of the MDC₉₀ are presented in Table 2.

Dose-dependent differences (higher average thresholds associated with larger differences) were found at the tibialis anterior muscle (slope of 0.12x, R² = 0.301) and the extensor carpi radialis brevis muscle (slope of 0.15x, R² = 0.253) using the Somedic algometer. Using the Wagner algometer, dose-dependent differences were present at the paraspinal muscles of C5-C6 (slope of 0.17x, R² = 0.225). Measurements at the rectus femoris muscle showed the lowest dose-dependent differences (Figs. 3 and 4).

The reliability was good to excellent at all locations with both algometers. The ICCs_1,1 for the Somedic algometer ranged from 0.89 to 0.93, and for the Wagner algometer from 0.87 to 0.95 (Fig. 2).

Exploration of individuals’ reproducibility

Individual within-session agreement expressed as SEM ranged from 24 to 120 kPa, depending on the rater, location and algometer. Consequently, the maximal individual offset compared to the group findings was −26 and +46 kPa (both on the rectus femoris muscle). The CV ranged from 5.1% to 21.3%, with maximal differences compared to the group findings of –6.0 to +9.3 percentage points (both on the rectus femoris muscle).

Individual between-session agreement expressed as SEM, ranged from 15 to 115 kPa, depending on the rater, location and algometer. Compared to the group findings, the maximal offset was −55 and +39 kPa (both on the tibialis anterior muscle). The CV ranged from 3.6% to 26.5%, and differences ranged from −9.5 to +6.6 percentage points compared to the group findings. For example, on the tibialis anterior muscle using the Wagner algometer, rater 3 had a CV of 3.6% (95% CI [2.4–7.5%], on an average PPT of 595 kPa) and the CV of rater 6 was 17.8% (95% CI [13.3–27.6%], on an average PPT of 645 kPa). Overall, most raters showed acceptable agreement scores, only rater 9 scored a CV of 26.5% (95% CI [18.3–50.8%], on an average PPT of 304 kPa) on the extensor carpi radialis brevis muscle. In general, the extensor carpi radialis brevis muscle showed the most unfavourable agreement scores, probably due to the smaller size increasing the complexity of the measurement (e.g., more difficult to localise and more difficult to keep the algometer on the muscle during the measurement). All individual rater reproducibility parameters are shown in Fig. 5 and are presented in Appendices A and B.

Between-session agreement and reliability were comparable for males and females except for the tibialis anterior muscle using the Wagner algometer (Appendix C).