The effectiveness of different attentional foci on the acquisition of sport-specific motor skills in healthy adults: a systematic review with network meta-analysis

Design

This study was a systematic review with network meta-analysis. We followed the guidelines of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al., 2022) and structured the reporting according to the PRISMA statement for network meta-analyses (Hutton, Catala-Lopez & Moher, 2016). A protocol of this review was published in the OSF registries (Favre-Bulle et al., 2022).

Searches

The search was performed on four electronic bibliographic databases; CINAHL Complete, Embase, MEDLINE (OVID) and Cochrane CENTRAL. The search strategy was developed and finalised on MEDLINE and then applied to the other databases mentioned above. Three search concepts were used to construct the search strategy. These comprised the i) population (i.e., people learning or training a sport-specific motor task), ii) interventions (i.e., different attentional foci), iii) outcomes (i.e., outcomes used to assess performance and learning) and a search filter for RCTs (Higgins et al., 2022). For each of the search concepts multiple free text search terms were identified and combined with subject headings (i.e., MESH terms). The search strategy for MEDLINE is presented in Appendix A1.

Condition or domain being studied

Acquisition of a sport-specific motor task.

Participants

Studies needed to report on adults (age between 18 and 65 years) learning or training a specific sport. There were no restrictions on the type of sport task or the level of experience in the sport. Several exclusion criteria were applied and the following populations were excluded: i) individuals with an injury or chronic illness, ii) individuals undergoing rehabilitation, iii) individuals over the age of 65 and minors.

Interventions

The sport-specific task had to be trained using at least one specific focus of attention. Multiple attentional foci could be compared within a study. Examples of eligible attentional foci were an internal or external attentional focus. In IFA, attention is directed to the body, whereas in EFA the attentional focus is directed to the movement effect (Wulf, 2013). This systematic literature review also included studies investigating other attentional foci such as a mixed attentional focus. A mixed attentional focus may comprise a combination of internal and external focused instructions. For example, an IFA is used in the preparation phase and an EFA is used in the execution phase of a motor skill (Connor et al., 2019).

Comparator

The included studies needed to have a control group. The control group could have a specific focus of attention, but another control intervention or a sham intervention was also possible.

Only studies with a randomized controlled trial design were included. Studies based on a cross-over design were also eligible for inclusion under two conditions: i) there was a random group allocation and ii) participants received only the allocated intervention for the specified cross-over period. For example, studies were excluded, if participants practiced a task with different attentional foci in the same period.

Main outcomes

The main outcomes of this review were the performance and learning of a specific sport-specific motor skill. Different outcome measures were allowed, as different sports were included. If a study reported multiple outcome measures for a sport-specific task, the review team reached a consensus on which outcome measure to use. The following factors played a role in the decision; i) homogeneity in terms of outcome measures of the included studies and ii) which outcome measure most accurately measured the performance and learning of the trained motor task. In order to prevent a unit-of-analysis error in our meta-analysis, it was determined that a single outcome measure should be included per study. In particular, the objective was to prevent the double counting of studies (Senn, 2009), ensuring that each participant’s data is represented only once in any given meta-analysis for each outcome assessed. Consequently, the most suitable outcome measure was selected for each study endpoint.

Performance at post-acquisition was the main outcome of this review and was assessed immediately after the intervention (i.e., training period) using post-acquisition tests. Two additional endpoints were analysed as indicators of learning. First, performance was assessed with retention tests. We used a minimal retention period of 24 h between the training period and the assessment of performance. In addition, transfer tests were used to analyse the performance on different tasks or conditions (Schmidt & Lee, 2014).

Data selection

The study selection was carried out in three steps. 1) All identified datasets were searched electronically for duplicates using the application Zotero (Guimarães et al., 2022); 2) The titles and abstracts of the remaining datasets were screened independently by two reviewers (SN, EFB). This step was performed using the software RAYYAN (Ouzzani et al., 2016). In a final step, the full text articles of the remaining datasets were screened independently by two reviewers (SN, EFB). In case of disagreements, the decisions were discussed jointly and, if necessary, resolved by a third person (KMS or CS).

Data extraction

Two examiners (SN, EFB) independently extracted relevant data into an electronic data extraction form. In case of missing data, the authors of the primary study were contacted by mail and asked for the data concerned. If data was not available in numeric form we used the application “WebPlotDigitizer” to extract data from figures (Drevon, Fursa & Malcolm, 2017).

Assessment of methodological quality

The PEDro scale was used to evaluate methodological quality of the included studies (de Morton, 2009). This process was carried out by two independent reviewers (SN, EFB). Conflicts were resolved by discussion within the review team.

Strategy for data synthesis

The data synthesis followed the guidelines of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins et al., 2022). The primary endpoint for assessing the effectiveness of the comparisons was at the end of the intervention (i.e., performance measured with post-acquisition tests). Secondary analyses were conducted using data from i) the longest available retention test and ii) transfer tests.

Analysis of the data

The analysis was performed using the statistical software R (R Core Team, 2023). A frequentist network meta-analysis was performed using the R package “netmeta” (Rücker et al., 2023). The transitivity assumption was examined for all studies included in the network meta-analysis (Salanti et al., 2014). The transitivity assumption is essential for network meta-analyses and the incorporation of indirect comparisons. Indirect observations provide observational evidence, but these may also contain biases and confounding. In order for an indirect comparison to be considered valid, it is necessary that the included studies exhibit, on average, comparability in all significant aspects, with the exception of the specific interventions being compared (Chaimani et al., 2023). In a more practical sense, studies that provide direct evidence (A against C and B against C) should exhibit a similar distribution of key aspects, including participants’ characteristics and methodological aspects. It is only then that the evidence of the indirect comparison (A against B) can be considered valid (Chaimani et al., 2017). Within this study, this was checked in two stages: Firstly, we investigated whether important effect modifiers were sufficiently similar between studies. For this we checked whether the following characteristics were similarly distributed: skill level at baseline, sample size and methodological quality measured with the Pedro scale. Secondly, we checked whether all interventions were meaningful to all participants. This means that any participant could theoretically have received any treatment in the network of interventions (Chaimani et al., 2017).

Regarding the relatively high degree of heterogeneity of included interventions, a random effects model was used for the network meta-analysis. The network of interventions was visualised using network plots. Within the plot, each intervention is represented as a circle. Circles are connected if there was at least one pairwise comparison. The network was analysed with regard to the presence of open and closed loops. The term “closed loop” is employed to describe comparisons with multiple pathways, including direct and indirect evidence. The term “open loop” is employed to describe comparisons that do not include direct evidence (Antoniou et al., 2019). Intervention effects were analysed as standardised mean differences (SMD). Effect sizes were classified as suggested by Cohen (1992). That is, 0.2 was considered a small, 0.5 a moderate and 0.8 a large effect. Effect estimates were presented in forest plots. We set the comparator to the intervention with the lowest estimated effectiveness. Direct and network estimates were compared and tabulated in a netleague table. P-Scores were used to produce a treatment ranking of the included interventions (Rücker & Schwarzer, 2015). P-Scores were calculated based on the observed effect sizes and corresponding standard errors, ranging from 0 to 1 on a continuous scale, with higher values indicating higher rankings. Heterogeneity was explored using Higgins’ I2 statistics and classified using the reference values from the Cochrane Handbook for systematic reviews of interventions (Higgins et al., 2022). In addition, we explored heterogeneity within designs and inconsistency between designs. Furthermore, inconsistency was visualised using a net heat plot or if not available with net splitting (comparison of direct and indirect effect estimates). A possible publication bias (i.e., the chance that studies with statistically significant findings are more likely to be published) was explored with comparison-adjusted funnel plots (Salanti et al., 2014).

Sensitivity analyses

Heterogeneity was explored with the help of meta-regressions. We used the meta package for these analyses (Schwarzer, Carpenter & Rücker, 2015). In a first step we performed pair-wise meta-analyses of the comparisons with the largest heterogeneity. A random-effects model was employed, with the restricted maximum likelihood ratio selected to estimate heterogeneity variance. In the second step, a mixed-effects model was utilised for the meta-regressions. We explored the influence of the variables age of participants, number of trials performed during the acquisition phase and methodological quality (i.e., measured with the number of items on the PEDro scale) on the effect estimates of the meta-analyses. Furthermore, we investigated the impact of removing studies with disparate effect sizes on the heterogeneity in the network meta-analysis.

Findings of the search

The search on the four databases resulted in a total of 5,031 identified records. After exclusion of duplicates, 3,952 records remained, which were screened for title and abstract. Of these, 3,872 records did not meet the criteria for further processing. Of the resulting 80 records, the full texts of 79 were identified. Subsequently, the reviewers screened the full texts for the specific inclusion and exclusion criteria, excluding 67 records. Finally, 12 studies were included in the review. The study selection process is shown in Fig. 1.

Included studies

A total of 566 subjects were included in the systematic review. Badminton (Abedanzadeh, Becker & Mousavi, 2022), golf (Aiken & Becker, 2022; Land, Frank & Schack, 2014; Munzert, Maurer & Reiser, 2014; Poolton et al., 2006; Wulf, Lauterbach & Toole, 1999), basketball (Al-Abood et al., 2002; Benjaminse et al., 2017), volleyball (Rostami et al., 2020), standing long jump (Yamada, Raisbeck & Porter, 2021), soccer (Makaruk et al., 2019), and dart throwing (Marchant et al., 2009) were the sports trained. The attentional focus strategies used in the study interventions were: EFA, IFA, holistic focus and switching focus of attention. In addition, there were variations of an EFA such as a dynamic EFA (i.e., instructions were used to focus the attention on the movement dynamics of a video model) and an imagined EFA (i.e., instructions were focused on an imagined EFA). All interventions and intervention labels are presented in Table 1. A post-acquisition endpoint was administered in all experiments. The endpoint retention test was performed in nine experiments. A transfer test was identified in six experiments. All study characteristics are listed in Table 2.

Methodological quality

For the following PEDro items, all studies included in the syntheses were considered at low risk of bias: random allocation, baseline comparability, outcomes of >85% of subjects, intention-to-treat analysis, between group comparison. Apart from Rostami et al. (2020), all studies received a high risk of bias for concealed allocation and blinding of therapists. In addition, all studies had a high risk of bias for the criterion subject blinding. The methodological quality is shown by means of a percentage diagram in Fig. 2.

Analysis of effectiveness

To test for transitivity, we checked whether potential effect modifiers (i.e., sample size, skill level and study quality) were similarly distributed across the comparisons. For all comparisons the sample size was <100 participants (mean sample size was 52 participants), the smallest average sample size was 45 (EFA vs holistic, IFA vs holistic) and the largest average sample size was 79 (EFA vs switching focus, IFA vs switching focus). The majority of participants were novices in all comparisons. For study quality, the average number of items with problems on the PEDro scale was 3.3. This ranged from an average of 2.75 items to four items.

Post-acquisition test

Ten studies with a total of k = 11 samples were included for the post-acquisition endpoint. In total m = 22 pairwise comparisons with n = 5 treatments and d = 5 different designs were available. The distribution of the network is presented in Fig. 3. The network consisted mainly of closed loops, open loops were found for the comparisons control vs switching focus” and “holistic focus vs switching focus” (i.e., for these comparisons only indirect evidence was available). The most common comparison was “EFA” vs “IFA” with m = 10 comparisons. The network plot with the number of comparisons is presented in Fig. 3A.

The network analysis revealed that, in comparison to the control group, “EFA” had the largest effect on performance measures at post-acquisition testing (SMD: 0.99; 95% CI [0.4 to 1.57]; p: 0.001). The least effective intervention compared to control was “IFA” (SMD: 0.11; 95% CI [−0.49 to 0.71]; p: 0.72). A forest plot of the effectiveness of all interventions at the post-acquisition endpoint with the control intervention set as the comparator is presented in Fig. 4.

The following treatment ranking was obtained based on the P-scores (from highest to lowest): “EFA”, “switching focus”, “holistic focus”, “IFA” and “control” (Appendix A2).

Heterogeneity in this analysis was substantial (I²: 71.3%). Further exploration showed that the heterogeneity could be explained by heterogeneity within designs (i.e., direct comparisons with Q: 32.59 and p: 0.001). The design with the highest contribution to the heterogeneity was “EFA” vs “IFA” (Q: 18.27 and p: 0.003). Between designs the inconsistency was not statistically significant (Q: 8.31 and p: 0.14). A netleague table comprising the direct effect estimates and the network estimates is presented in Appendix A3. The largest differences were observed for the comparisons with only one study presenting direct evidence (i.e., “IFA” vs “holistic focus” and “IFA” vs “switching focus”). In addition, a heat plot showed that overall, the inconsistency was small in the analysed network (Appendix A4). Most studies were symmetrical distributed in a comparison adjusted funnel plot (Appendix A5).

Two studies reporting on different versions of an EFA at post-acquisition and could not be integrated into the network meta-analysis. Yamada, Raisbeck & Porter (2021) demonstrated that an imagined EFA and an EFA yielded comparable results for this endpoint. Al-Abood et al. (2002) showed similar effectiveness of a dynamic EFA compared to an EFA.

Retention test

Seven studies (k = 7) were included for the retention test endpoint. In total, m = 13 comparisons with n = 5 treatments and d = 4 designs were available in the dataset. The network is presented in Fig. 3B. The network comprised mainly of closed loops. For the comparisons “switching focus vs control” and “holistic focus vs switching focus” only indirect evidence was available. With regard to the remaining comparisons, at least one study provided direct evidence. The most frequent comparison was “EFA” vs “IFA” with m = 7 comparisons.

The results of the network meta-analysis showed that the intervention with the largest effect on performance measures at retention testing was “holistic focus” with a moderate to large effect size (SMD: 0.75; 95% CI [−0.1 to 1.6]; p: 0.08) followed by “EFA” with a moderate effect size (SMD: 0.56; 95% CI [0.20–0.92]; p: 0.002) when compared to IFA (Fig. 5). P-Scores indicated the following treatment ranking at retention testing: “holistic focus” (P-Score: 0.75), “EFA” (P-Score: 0.63), “switching focus” (P-Score: 0.57), “control” (P-Score: 0.5), “IFA” (P-Score: 0.06). Heterogeneity was moderate in this network meta-analysis (I²: 44.5%). Heterogeneity within designs was not statistically significant (Q: 3.72 and p: 0.29). Similar results were found for between designs inconsistency (Q: 7.22 and p: 0.06). A comparison of direct evidence and network estimates is presented in Appendix A6. Overall network and direct evidence estimates were similar. Net splitting did not show any significant differences between direct and indirect effect estimates (Appendix A7). A comparison adjusted funnel plot did not show asymmetry (Appendix A8).

Transfer test

The network meta-analysis for the transfer endpoint comprised k = 4 studies, m = 8 comparisons, n = 4 interventions, and d = 3 designs. The network plot is presented in Fig. 3C. The network was comprised of closed loops, with the exception of the comparison between a holistic focus and a switching focus. Most comparisons were found for the comparison “EFA” vs “IFA (m = 4).

The most effective intervention at transfer testing (i.e., in comparison to IFA) was “holistic focus” with a large effect size (SMD: 1.16; 95% CI [0.47–1.86]; p: 0.001), followed by an “EFA” (SMD: 0.35; 95% CI [0.02–0.68]; p: 0.04) (Fig. 6). The following treatment ranking was obtained: “holistic focus” (P-Score: 0.99), “EFA” (P-Score: 0.57), “switching focus” (P-Score: 0.35) and “IFA” (P-Score: 0.08).

Heterogeneity in the network meta-analysis at transfer testing was classified as low (I²: 0%). Heterogeneity within and between designs was not statistically significant (Q: 0.37 and p: 0.54 respectively Q: 1.53 and p: 0.46). A netleague table presenting direct and network estimates is presented in Appendix A9. Net splitting did not show any statistically significant differences between direct and indirect effect estimates (Appendix A10).

Sensitivity analyses

In order to investigate the heterogeneity at the post-acquisition endpoint several analyses were performed. The majority of heterogeneity was identified in the comparison EFA vs IFA. We explored the influence of the following potential moderator variables: age of the participants, number of trials during acquisition and methodological quality with the number of points on the PEDro scale. In examining the influence of age on the post-acquisition endpoint, the meta-regression analysis yielded an R² value of 0%, indicating that age did not account for any of the variance in the effect sizes across studies. The model’s p-value was 0.48, suggesting that age was not a statistically significant predictor of the outcomes. The estimated effect size for age was 0.07, with a 95% confidence interval ranging from [−0.176 to 0.33]. Similar results were found for the variables number of trials (model: R²: 0%, p: 0.48, effect estimate: −0.001 (95% CI [−0.005 to 0.003]) and methodological quality (model: R²: 0%, p: 0.53, effect estimate: 0.33 (95% CI [−0.86 to 1.52]).

Another sensitivity analysis was performed in an attempt to decrease heterogeneity within the post-acquisition endpoint network meta-analysis. Within this analysis we excluded a single study from the network analysis. The study of Wulf, Lauterbach & Toole (1999) showed a very large effect (SMD: 2.51; SE: 0.58) in favour of an EFA and the effect estimates differed considerably from the other studies in the analysis. The exclusion of this study resulted in a slight reduction in heterogeneity (I²: 66% vs 73%) (Appendix A11).

Limitations

The results of the present study should only be interpreted with caution due to the heterogeneity in the analyses. This was especially the case for the analysis of the post-acquisition endpoint. There are several possible reasons for the moderate to considerable heterogeneity: i) the variability of the sample sizes of the included studies; ii) the differences in the population in the included studies, such as different sports or different skill levels, various task complexity and motor skills practiced; iii) methodological heterogeneity. However, we tried to eliminate this source of heterogeneity with strict selection criteria. Only studies with randomised controlled trial design were included in this systematic review. In addition, the analyses of the methodological quality showed similar ratings on the PEDro items and most studies had lower ratings in the same categories.

Another limitation of this review was the limited sample size of the included studies. Most of the studies had relatively small sample sizes and a small study bias cannot be excluded (Sterne, Egger & Smith, 2001). Furthermore, the data available did not permit an assessment of the task-specific effects.

Future research should consist of larger study samples, investigating new attentional focus strategies, and clarifying the optimal timing and combination of different foci during skill acquisition and motor learning. In addition, further research should focus on the strategies identified, such as holistic focus and imagined EFA, using larger sample sizes and randomised controlled trial designs. In practical terms, the current evidence strongly supports the recommendation of using an EFA for individuals training in specific sports.

Implications

The present study has several implications for research. First, our results indicate that despite a large body of evidence, there is still a lack of larger studies (i.e., recruiting more than 100 participants) investigating this motor learning principle. Second, currently, studies using a cross-over design often do not specify the time interval between two different training periods (i.e., the “wash-out period”), which limits any interpretation due to possible carry-over effects. Moreover, our findings supporting the promising effects obtained with specific attentional strategies such as “holistic focus” or “imagined EFA” motivate further follow-up studies of these methods in sports, but also in other populations (e.g., neurological patients).

The implications for practice are as follows: crucially, the present findings indicate that there is sufficient evidence to recommend the use of an EFA for the acquisition of sport-specific motor tasks. An IFA showed lower effectiveness compared to an EFA on all three endpoints and should therefore only be used with caution in this specific population (i.e., healthy adults training a specific sport).