Evidence of validity and reliability of Jumpo 2 and MyJump 2 for estimating vertical jump variables

Introduction

The countermovement jump (CMJ) and squat jump (SJ) are not only exercises that are used in training, but they are also movements that are widely used to monitor neuromuscular performance (van Hooren & Zolotarjova, 2017). Due to the inherent differences between these movements, the CMJ usually results in greater performance (i.e., ∼20% greater jump heights) mostly due the effects of the stretch-shortening cycle (van Hooren & Zolotarjova, 2017). As the SJ does not make use of the stretch-shortening cycle, the CMJ and SJ are often both assessed to provide a more holistic view of lower body explosive strength performance. By using a force plate to assess CMJ or SJ performance, several variables can be obtained that can provide useful information regarding the mechanical capabilities of the neuromuscular system. Although force plates are, in fact, becoming more accessible than they were in the past, the high cost associated with a force plate system still presents a major barrier to their use among the general public.

As an alternative, smartphone applications (app) have been developed that make use of mobile phone cameras instead of large expensive force plates. One of the most popular apps, MyJump 2 which operates on the iOS system, has been shown to provide valid and reliable measures of the height of CMJ and SJ (estimated from flight time) (Balsalobre-Fernández, Glaister & Lockey, 2015; Coswig et al., 2019; Cruvinel-Cabral et al., 2018; Brooks, Benson & Bruce, 2018). More recently, other studies found valid and reliable measures of both CMJ and SJ flight time when using the Jumpo app, which runs on an Android operating system, allowing for a far broader user base than apps on iOS (Vieira et al., 2021; Azevedo et al., 2019). As these apps directly measure flight time, it is logical that this data can be used to then estimate jump height. However, these apps take things a step further and claim to calculate the mean values of force, velocity, and power produced during the propulsive phase (i.e., the ascendant aspect of the movement) of the vertical jump (Samozino et al., 2008).

These measures can be highly informative for those seeking to analyze the force-velocity capabilities of the lower limbs. However, a recent study has questioned the validity of Samozino’s method of estimating force, velocity, and power (Hicks, Drummond & Williams, 2021). To briefly explain, Hicks, Drummond & Williams (2021) observed that Samozino’s method overestimated force and underestimated velocity and power of the CMJ compared to the force plate. However, it is important to note that the proposed method by Samozino et al. (2008) was developed specifically for the SJ exercise, and since a difference exists between CMJ and SJ performance, a study applying Samozino’s method in both CMJ and SJ is required to show whether the jump type might play a role in this discrepancy.

Lastly, the validated iOS MyJump app has recently released an Android version, and the Jumpo app released its second version on Android, but neither the validity nor the reliability of these apps have been determined. Android is the most popular operating system and, unlike iOS, runs on devices from different companies, which implies that these devices may have different hardware and software characteristics like camera resolution, file storage, etc. Therefore, the purpose of the presented study was to investigate the concurrent validity and test-retest reliability of the Jumpo 2 and MyJump 2, both running on an Android system, for estimating jump height and the mean values of force, velocity, and power produced during CMJ and SJ.

Materials & Methods

Study design

The participants visited the laboratory on three occasions (2 to 7 days apart). Firstly, anthropometric measurements were taken, such as the vertical displacement of the center of mass during the propulsion phase of the vertical jumps (SCoM; Fig. 1) (Samozino et al., 2008). In addition, the participants were familiarized with CMJ and SJ jumping technique and procedures, and subjects were considered to be familiarized when there was <1 cm difference in jump height between consecutive jumps of each jump type. In the following two visits, they completed ∼10 min of a standardized warm-up protocol composed of traditional back squats (50% body mass) in a Smith Machine followed by five vertical jumps of progressive effort (i.e., 20, 40, 60, 80 and 100% of the perceived maximum). In a quasi-randomized order, participants completed four repetitions of each jump type (countermovement jump [CMJ] and squat jump [SJ]), with all four jumps of one type followed by all four jumps of the other type. Jumps were performed on a 101 × 76 cm force plate (Accupower Portable Force Plate; AMTI, Watertown, MA, EUA) while being recorded by a slow-motion camera (240 fps) on a Samsung Galaxy S7 phone (model SM - G930F). Each jump was separated by 1 min of rest, and the same order of the jumps was kept for the re-test session for each participant.

Statistical analyses

The average values (from four jumps) for each variable were analysed. Several analyses were conducted to determine the concurrent validity of both apps and the test-retest reliability of all devices. Comparisons between data from both apps and the force plate were made using one way ANOVA since all assumptions were met. When a significant difference was detected, the Tukey HSD post-hoc test was applied for multiple comparisons.

Regarding the concurrent validity, a regression analysis assessed the relationship between jump performance variables obtained from the force plate and Jumpo 2 app and again between the force plate and the MyJump 2 app (Hopkins, 2004; Hopkins, 2015). The validity correlations obtained from the regression analysis were interpreted as excellent (>0.995), very good (0.995–0.950), good (0.949–0.850), poor (0.849–0.700), very poor (0.699–0.450), and impractical (<0.450) (Hopkins, 2015). Raw and percentage (as a coefficient of variation –CV%) values of typical error of estimates were calculated and are displayed in Fig. 2. In addition, bias analysis (i.e., uniformity of error) was performed by plotting residuals (predicted values minus “real” values obtained from force plate) against predicted values and then calculating a Pearson correlation between the residuals and predicted values.

Regarding the test-retest reliability, intraclass correlation coefficient (ICC) and typical error of measurement as a CV % were calculated. CV % were interpreted as good (≤ 5%), moderate (5–10%), and poor (>10%) (Hopkins, 2015), while ICC were interpreted as poor (<0.500), moderate (0.500–0.750), good (0.749–0.900), and excellent (>0.900) (Koo & Li, 2016). Variables were assumed as “reliable” when the CV% was ≤ 10% and ICC was ≥ 0.75. IBM SPSS Statistics software (version 26; IBM Co., Chicago, IL, USA) was used for inferential statistics analyses, and a custom-made spreadsheet was used for validity and reliability (Hopkins, 2015).

Results

The regression analyses of jump variables obtained from the force plate and both apps are displayed in Fig. 2. Considering both CMJ and SJ, the validity correlations were “very good” for jump height for both apps against the force plate (Figs. 2A–2B). In contrast, we observed “poor” correlations for force (Figs. 2C–2D) and “very poor” correlations for velocity and power (Figs. 2E–2H). To note, CMJ demonstrated “good” to “very good” correlations for all the measured variables, except velocity, which presented a “poor” correlation. SJ demonstrated a “very good” correlation only for jump height, while force was “poor” and both velocity and power were “impractical” (r < 0.410). No evidence of heterocedascity (i.e., non-uniformity of error) was observed for any variables since the Pearson correlations approached zero (slope < 0.01).

Comparisons between measures of the vertical jump obtained with the force plate and both apps are presented at Table 1. We found a “large” difference (p =  < 0.001; η² = 0.569) between the mean velocity of the CMJ measured with the force plate compared with those measures that were estimated from both apps. We did not observe any other differences in the remaining variables.

The intra-rater reliability of the vertical jump measures obtained with the force plate and both apps is displayed at Fig. 3 and Table S1. Our results suggest that the majority of the measures show acceptable scores of reliability, and the measures obtained from CMJ were more likely to be more reliable than SJ. Note that the confidence intervals from SJ data more often extended past the CV and/or ICC limits compared to CMJ data.

Discussion

We investigated the concurrent validity and test-retest reliability of the Jumpo 2 and MyJump 2 apps for estimating jump height and mean values of force, velocity, and power generated during the propulsive phase of CMJ and SJ. No differences between Jumpo 2 and MyJump 2 were identified, but some large differences between the force plate and both apps were found (Fig. 2 and Table 1). Additionally, both apps provide a valid estimate of the jump height and mean propulsive force of the CMJ, whereas only jump height can be considered valid from the SJ (Fig. 2). Furthermore, the apps provide reliable estimates compared to the force plate (i.e., criterion method), but the CMJ is more likely to have better scores of reliability than the SJ (Fig. 3).

The present findings corroborate with several previous studies demonstrating that the iOS version of the MyJump app (Balsalobre-Fernández, Glaister & Lockey, 2015; Coswig et al., 2019; Cruvinel-Cabral et al., 2018; Brooks, Benson & Bruce, 2018) and the original Jumpo app for Android (Vieira et al., 2021; Azevedo et al., 2019) provide valid measures of jump height estimated from flight time. Nevertheless, to our knowledge, this is the first study to demonstrate that mean force can also be accurately estimated from the CMJ using smartphone apps. Although this may be true, we found a large discrepancy between the mean propulsive velocity (and as consequence mean propulsive power) measured with the force plate against the estimated mean velocity with both the apps (Fig. 2 and Table 1). Impractical (r² <0.001) and poor (r² = ∼0.64) validity correlations were observed for both SJ and CMJ, respectively (Figs. 2E and 2F). Furthermore, the apps underestimated mean velocity during the CMJ (Table 1) by ∼0.26 m/s, which is quite substantial considering the normal mean propulsive velocity of jumping in this population. With this in mind, it is important to note that both apps estimate mean velocity (see Eq. (2)), which would ideally require a constant CoM acceleration and an accurate measure of jump height (Samozino et al., 2008). However, these assumptions might not be true because the CoM likely decelerates a few milliseconds before takeoff (probably due to the addition of the shank and foot segments to the accelerating mass (McMahon et al., 2018)) and the estimated jump height from flight time is prone to overestimate the peak of vertical displacement of the CoM (Aragon-Vargas, 2000). In support of this, a recent study (Hicks, Drummond & Williams, 2021) also found discrepancies in estimating CMJ mean velocity using Samozino’s method. Hicks, Drummond & Williams (2021) observed that Samozino’s method provided a poor validity correlation (r² = 0.64) and underestimated mean velocity by 0.26 m/s (16.4%) compared with the force plate. Note that we have found the same results for the CMJ, but even worse validity correlations for the SJ. In this regard, we could speculate that since CMJ performance is usually better than SJ performance, the SJ is more prone to be affected by the previously mentioned source of errors (lower signal to noise ratio).

Regardless of the jump type and the device used, there was an acceptable level of test-retest reliability for all the measured variables (ICC ≥ 0.75 and CV <10%, Fig. 3). It was also noted that the CMJ (ICC ≥ 0.85 and CV ≤ 4.6%) demonstrated slightly better scores of reliability than the SJ (ICC ≥ 0.84 and CV ≤ 8.5%), which corroborates with previous studies demonstrating that the CMJ produces better reliability scores than the SJ (Vieira et al., 2021; Gallardo-Fuentes et al., 2016). Furthermore, both of the apps and the force plate demonstrated similar scores of reliability, which is an important finding considering the ecological validity of our methods and having an independent observer analyze the data without any previous experience in video analysis.

Considering all of the above, the present results have practical application for those seeking a simple approach to monitor and measure relevant performance variables with inexpensive mobile apps. Our findings revealed that the Jumpo 2 and MyJump 2 both provide reliable estimates of jump height and mean values of force, velocity, and power produced during the propulsive phase of both CMJ and SJ. However, the observed differences between force plate data and apps values for velocity and power of the CMJ as well as from all those variables except jump height from SJ prevent the users from confidently applying these measures in real life, for instance, to analyze the force-velocity capabilities during vertical jumps. It is worth mentioning that our sample size was small for those metrics beyond jump height (CMJ and SJ) and CMJ mean force due to their poor-to-impractical validity correlations. Furthermore, the present study included only physically active college students and further testing in other populations (e.g., athletes, elderly, and sedentary individuals) is warranted before using these apps in those populations who may have better or worse jump mechanics and flight paths, as flight time is the main variable of interest when using these apps.

Conclusions

Jumpo 2 and MyJump 2 mobile apps running on the Android system are reliable tools to monitor vertical jump performance variables, presenting scores of reliability similar to those observed with a force plate (i.e., criterion equipment). Users should keep in mind that both apps provide valid measures of the height of CMJ and SJ (estimated from flight time) as well as the force produced during CMJ. However, a difference exists between velocity (and as consequence, power) when comparing force plate data against the estimated data from these apps.

Supplemental Information