Comparison between Mother, ActiGraph wGT3X-BT, and a hand tally for measuring steps at various walking speeds under controlled conditions

Ethics statement

Ethical approval of the research protocol was not needed according to The North Denmark Region Committee on Health Research Ethics. Written informed consent was signed by all participants prior to the study.

Sample size and raters

Sample size was determined to be 30 participants using large sample case (Hogg & Tanis, 1996). Two raters performed the hand tallying and conducted the treadmill test whilst two different raters performed the data treatment without being part of the data collection.

Participants

Thirty-one healthy students were recruited from Aalborg University, Denmark (male n = 15, female n = 16). Data was collected from March to April 2015.

The inclusion criteria were: (i) age of 18 years or above, (ii) no self-reported health problems evaluated by the Physical Activity Readiness Questionnaire (PAR-Q) (Shephard, 1988), (iii) ability to walk without walking aids, (iv) ability to walk continuously for 10 min on a treadmill, and (v) ability to read, understand and speak Danish and English. The fifth criterion was chosen to ensure that participants understood the instructions and the PAR-Q, which was in English, as no Danish translation was available.

The exclusion criteria were: (i) pregnancy (self-reported), (ii) BMI ≥ 30 kg/m², and (iii) neurological diseases (self-reported). 

Thirty-one healthy participants were recruited for the study. One participant was excluded due to a BMI > 30 kg/m². Participants did not report any impairments or morbidities potentially interfering with the assessment. See Table 1 for participant demographics.

Mother

Mother (Sen.se, Paris, France) is a triaxial accelerometer released in March 2014 by Sen.se (Paris, France). The device consists of a hub (Mother), up to 24 sensors (Motion Cookies), and a software application (the Senseboard) (Sen.se, Paris, France). User access is gained through the Senseboard, which is a collection of different applications developed by Sen.se. The dimensions of a Motion Cookie are 5.0*2.2*0.4 cm with a weight of 6 grams (Sen.se, Paris, France). The sample rate is fixed at 25 Hz. The accelerometer has a dynamic range of ±2G and a precision of 12 bit (Alain Romanet, e-mail correspondence with Sen.se, March 9th 2016). In this study the application Walk (Sen.se, Paris, France) was selected for measuring steps. Data from the Motion Cookie is uploaded to Mother every 5 min (Franck Biehler, e-mail correspondence with Sen.se, March 23rd 2015) in fractions of varying durations.

ActiGraph wGT3X-BT

ActiGraph wGT3X-BT (ActiGraph LLC, Pensacola, FL, USA) is a triaxial accelerometer and one of the most commonly used devices for assessing PA (Crouter et al., 2013). The dimensions of the sensor are 4.6*3.3*1.5 cm with a weight of 19 grams. A sample rate of 100 Hz was chosen (range 30–100 Hz). The accelerometer has a dynamic range of ±8G and a precision of 12 bit (ActiGraph LLC, Pensacola, FL, USA). Data is accessible by using the ActiLife Pro 6 software (ActiGraph LLC, Pensacola, FL, USA). After the recordings the sensor was connected to a computer through a mini-USB cable in order to upload the data. During initialization, information including subject name, gender, height, weight, race, limb and leg dominance is required, however random values were used as step counting is not affected by this information (ActiGraph LLC, Pensacola, FL, USA).

Accelerometer placement

The participants wore Mother’s Motion Cookie and ActiGraph simultaneously placed on an elastic belt above the right anterior superior iliac spine (SIAS) (0.5 cm. medially and laterally from the right SIAS, respectively) (Fig. 1). Hip placement has previously been found as the most precise single location placement of an accelerometer (Cleland et al., 2013). The placement of devices was randomized using a random number generator (http://www.random.org) (Randomization 0: Mother laterally, Randomization 1: ActiGraph laterally) to take possible placement related differences in validity into account. Randomization 0 was received by 17 participants and Randomization 1 was received by 13 participants.

Hand tally

A hand tally is considered the gold standard when measuring steps (Dijkstra et al., 2008; Fortune et al., 2014; Stemland et al., 2015). In this study, the application AGR Tally counter (ver. 1.0, Angel Garcia Rubio) was used for hand tallying. Steps were measured by tapping the screen of an iPhone 4s (Apple Inc., Infinite Loop Cupertino, California, USA).

Procedures of the treadmill test

The number of steps was obtained during a treadmill test where participants walked on a treadmill for two minutes at each of the three walking speeds; 3.2, 4.8, and 6.4 km/h. The inclination was set to 0°. The treadmill was preprogrammed to standardize the procedure for an in- and decrease of the walking speed. The two minutes included the time the treadmill in- and decreased the walking speed. The speeds were chosen in accordance with walking speeds chosen in three previous studies that investigated the validity of measuring steps under controlled conditions at various walking speeds (Steeves et al., 2011; De Cocker et al., 2012; Clemes et al., 2010). The test was conducted in the sports science laboratory at Aalborg University, Denmark. To take the inability of Mother to synchronize more often than every 5 min into consideration, the participants were asked to stand still for five minutes and 10 s before and after each walking speed in order to identify the walking session in the application programming interface (API).

Data Treatment

Data from Mother for each test were identified by examining the walking duration in the API. Even though the participants walked for exactly two minutes, which was confirmed by video recordings of the tests, walking sessions had durations ranging from 115–130 s. The output from the API showing the number of steps of the walking session was manually examined to identify equipment malfunctions such as missing steps. Accelerometer data from ActiGraph were downloaded using ActiLife 6 Pro software.

Video recordings of the treadmill test were used for hand tallying. The definition of a step was adopted by Dijkstra et al. (2008) and defined as “the first moment at which the heel of the foot for the initial step cleared the ground and the moment at which the foot of the closing step made completely contact with the floor”. The hand tallying procedure was double validated as the two raters hand tallied independently. The two raters had 100% agreement. The two raters responsible for data treatment and the statistical analyses had not been involved in the treadmill test nor the hand tallying.

Statistical analysis

All statistical analyses were performed using IBM SPSS (ver. 22, IBM Corporation, New York, United States) with a significance level of p < 0.05.

Normal distribution was examined based on the differences between the number of steps measured by Mother and the hand tally and ActiGraph and the hand tally using Q-Q plots and Shapiro–Wilk test. Q-Q plots were assessed and as data did not appear to be normally distributed, a Shapiro–Wilk test was performed and confirmed that data were non-normally distributed (p < 0.05) (DOI: 10.6084/m9.figshare.3814272.v1).

Wilcoxon’s signed ranks test was used to create a pairwise comparison to determine whether the devices and the hand tally measured the same number of steps. Means and standard deviations (SD) were computed for age, height, weight and BMI whilst medians and interquartile ranges (IQR) were computed for steps and differences in steps measured by the devices. A two-way random effects model (2.1), single measures, absolute agreement, and intraclass correlation coefficients (ICC) with 95% confidence intervals were used to express interrater reliability between the devices and the hand tally. ICC values >0.75 were interpreted excellent, 0.40–0.75 were interpreted fair to good and <0.40 were interpreted poor (Fleiss, 1999). To investigate the average error Root Mean Square error (RMSe) was calculated between each device and the hand tally. As data were non-normally distributed the nonparametric approach to presenting Bland–Altman plots was adopted and the median, the 2.5th, and the 97.5th percentiles were visualised in the plots (Gialamas et al., 2010).

Percent differences (PD) were calculated for between-instrument agreement (Mother vs. the hand tally and ActiGraph vs. the hand tally). The PD was calculated as $\frac{Absolutedifference}{handtallysteps}\cdot 100$. Any negative values were converted to a positive to calculate the absolute difference. Clinical relevance of potential under- or over-counting of steps by the two accelerometers compared to hand tally was determined using a 3% criterion, which was based on previous studies (Johnson et al., 2015; Holbrook, Barreira & Kang, 2009; Colley et al., 2013; Liu et al., 2015). PD ≤3% were considered clinically irrelevant.

Mother vs. the hand tally

The median of differences in steps between Mother and the hand tally at the different walking speeds were 2.5 steps (IQR = 5) at 3.2 km/h, 2 steps (IQR = 2) at 4.8 km/h and 3.5 steps (IQR = 6) at 6.4 km/h (Table 2). These are depicted in the Bland–Altman plot (Fig. 2). The median of total differences was 9.5 steps (IQR = 10).

Wilcoxon’s signed ranks test showed a significant difference in the number of steps measured by the hand tally vs. Mother at all walking speeds (p < 0.001) (Table 3).

The ICCs for Mother and the hand tally were all excellent ranging from 0.88 (0.51–0.96) at a speed of 3.2 km/h to 0.96 (0.72–0.99) at a speed of 4.8 km/h (Table 3). The RMSe ranged from 2.86 at 4.8 km/h to 5.50 at 3.2 km/h (Table 3). Mother had PDs ≤2.5% of the steps measured by the hand tally at all speeds (Table 3).

ActiGraph vs. the hand tally

The median of differences in steps between ActiGraph and the hand tally at the different walking speeds were 49.5 steps (IQR = 69) at 3.2 km/h, 4 steps (IQR = 5) at 4.8 km/h and 4 steps (IQR = 5) at 6.4 km/h (Table 2). These are depicted in the Bland–Altman plot (Fig. 3). The median of total differences was 59 steps (IQR = 77).

Wilcoxon’s signed ranks test showed a significant difference in the number of steps measured by the hand tally vs. ActiGraph at all walking speeds (p < 0.001) (Table 3).

The ICCs for ActiGraph and the hand tally were poor and fair to good ranging from 0.03 (−0.09–0.21) at a speed of 3.2 km/h to 0.64 (0.16–0.84) at a speed of 6.4 km/h (Table 3). The RMSe ranged from 8.80 at 6.4 km/h to 36.52 at 3.2 km/h (Table 3).

ActiGraph had PDs ≤26.7% of the steps measured by the hand tally (Table 3). ActiGraph was under-counting based on the 3% criterion at both 3.2 and 4.8 km/h.

Limitations

The participants of this study were a group of younger, healthy subjects who performed steps that were similar to the definition of a step by Dijkstra et al. (2008). However, patients may walk asymmetrically or without a swing phase which might provide different results. Healthy participants were chosen, as the purpose of this study was to test the validity under controlled conditions. Therefore, the results cannot be directly applied to any given patient group.

The walking sessions in the API of Mother had varying durations ranging from 115 to 130 s. The reason for this variation is unknown, however, it implies that the participants have either stopped walking prior to the end of the two minutes or they have continued moving even though the treadmill had stopped. This was, however, not indicated by the video recordings. Another explanation for the 115 s walking sessions could be that Mother stopped measuring as the participant slowed down before coming to a halt. As Mother measured durations both shorter and longer than the 120 s the walking session lasted, the inconsistencies in duration may have evened out.

Future work

This study investigated step measuring at a walking speed of 3.2 km/h as the lowest speed, but some patients may walk at an even slower pace, thus investigating the validity at lower walking speeds is highly relevant to determine the minimum speed at which Mother still provides measures of steps that have clinically irrelevant deviations from the actual number of steps. Future studies should also include testing in a semi-controlled environment and in free-living conditions and should also include participants with a larger BMI than included in this study as the waist circumference can influence precision due to tilting (Crouter, Schneider & Bassett, 2005).