The effects of high versus low talker variability and individual aptitude on phonetic training of Mandarin lexical tones

High variability L2 phonetic training for non-tonal contrasts

A substantial body of literature has explored whether phonetic training can be used to improve identification and discrimination of non-native phonetic contrasts in L2 learners. An early study by Strange & Dittmann (1984) attempted to train Japanese speakers on the English /r/-/l/ distinction, a phoneme contrast that does not exist in Japanese. Participants were trained on stimuli from a synthetic rock-lock continuum. The key result was that although performance increased both for trained and novel synthetic items, participants failed to show any improvement for naturally produced minimal pair items. Later research suggested that a key factor which prevented generalisation to natural speech tokens was a lack of variability in the training materials: Variability was present in the form of the ambiguous intermediate stimuli along the continuum, however, there was a single phonetic context and a single (synthesized) speaker. Logan, Lively & Pisoni (1991) also trained Japanese learners on the English /r/-/l/ contrast, but included multiple natural exemplars spoken by six speakers, with the target speech sounds appearing in a range of phonetic contexts. In contrast to Strange and Dittman, they found that participants successfully generalised both to new speakers and new words at test. This was the first study to indicate the importance of variability within the training materials. A follow up study by Lively, Logan & Pisoni (1993) provided further evidence for this by contrasting a condition with high variability (HV) input to one with low variability (LV) input in which the stimuli were spoken by a single speaker (although still exemplified in multiple phonetic contexts). Participants in the LV group improved during the training sessions but failed to generalise this learning to a new speaker.

Following Lively, Logan & Pisoni (1993) high variability phonetic training (HVPT) has become standard in L2 phonetic training. This methodology has been successfully extended to training a variety of contrasts in various languages such as learning of the English /u/-/ʊ/ distinction by Catalan/Spanish bilinguals (Aliaga-García & Mora, 2009), learning of the English /i/-/ɪ/ contrast by native Greek speakers (Giannakopoulou, Uther & Ylinen, 2013; Lengeris & Hazan, 2010), and learning of the English /w/-/v/ distinction by native German speakers (Iverson et al., 2008).

There is also some evidence that this type of perceptual training benefits production in addition to perception. Bradlow et al. (1999) found that production of the /r/-/l/ contrast improved in Japanese speakers following HVPT, with this improvement being retained even after 3 months. Similar improvement on the production of American English mid to low vowels by Japanese speakers following HVPT was also reported by Lambacher et al. (2005). However, the evidence here is mixed: A recent study by Alshangiti & Evans (2014) employed HVPT to train Arabic learners on non-native English vowel contrasts and found no improvements in production, although participants receiving additional explicit production training did show some limited improvement.

Although the studies reviewed above all used HVPT, only the original work by Lively, Logan & Pisoni (1993) directly contrasted the use of high and low variability materials. It is notable these seminal experiments used small samples (the tests of generalisation were administered to only three of the participants). Since then, few studies have explicitly contrasted high and low variability training. One such study was Sadakata & McQueen (2013), who trained native Dutch speakers with geminate and singleton variants of the Japanese fricative /s/. Participants were trained with either a limited set of words recorded by a single speaker (LV) or with a more variable set of words recorded by multiple speakers (HV). Both types of training led to increases in generalisation to untrained fricatives and speakers. However, in an identification task, the improvement was greater for participants receiving HV training than those receiving LV training. Similar results were reported by Wong (2014) who trained native Cantonese speakers with the English /e/-/æ/ contrast. Both LV (one speaker) and HV (six speakers) training lead to increased performance from pre- to post-test, but the improvement was greater for the HV group. This was found in tests of generalisation to new speakers and new items, and from perception to production. In contrast, a recent phonetic training study did not find the same benefit. Giannakopoulou et al. (2017) compared matched HV (four speakers) and LV (one speaker) training for adult and child (8-year-old) native Greek speakers who were trained on the English /i/-/ɪ/ contrast. This study did not show a benefit for HV compared to LV training in either age group, even for generalisation items. However, for adult participants, it is unclear the extent to which this was due to ceiling effects. To our knowledge, the only other previous studies that specifically manipulated variability during learning of non-native phonetic categories are those by Perrachione et al. (2011) and Sadakata & McQueen (2014), which both looked at the learning of lexical tone. We discuss these studies in more detail in the following section.

Although there is a relatively small evidence base regarding a benefit of high over low phonetic training for non-native phoneme categories, there is further evidence for this benefit in related areas of speech and language learning, specifically accent categorization and adaptation (Bradlow & Bent, 2008; Clopper & Pisoni, 2004), and L2 vocabulary learning (Barcroft & Sommers, 2005, 2014; Sommers & Barcroft, 2007, 2011). Benefits of HVPT are generally seen in tasks of generalisation, suggesting that exposure to variation across speakers and/or items boosts the ability to generalise across these dimensions. This intuitively sensible result is in line with the predictions of computational models in which irrelevant contextual/speaker identity cues compete with phonetically relevant cues, so that dissociation of these irrelevant cues is the key mechanism which underpins generalisation (Apfelbaum & McMurray, 2011; Ramscar & Baayen, 2013; Ramscar et al., 2010).

Phonetic training of L2 lexical tones

Each of the phonetic training studies discussed above involved training a segmental contrast (consonantal or vocalic). Lexical tone is another type of phonological contrast in some natural languages, whereby the pitch contour is used to distinguish lexical or grammatical meanings (Yip, 2002). For example, Mandarin Chinese has four lexical tones: level-tone (Tone 1), rising-tone (Tone 2), dipping-tone (Tone 3) and falling-tone (Tone 4). These pitch contours combine with syllables to distinguish meanings. For instance, the syllable ba combines with the four tones to mean: eight (bā, Tone 1), pluck (bá, Tone 2), grasp (bǎ, Tone 3) and father (bà, Tone 4). Each of these words thus forms a minimal pair with each of the others. Note that while non-tonal languages such as English use pitch information extensively for intonation (e.g. forming a question, or for emphasis), and that pitch plays a role in marking stress at the lexical level in (e.g. IMport/imPORT), this is quite different from a lexical tone system, causing difficulties for L2 learners of Mandarin.

The first study examining lexical tone training was conducted by Wang et al. (1999). A similar paradigm to that used by Logan, Lively & Pisoni (1991) was adopted using four speakers for training. Training materials were all real monosyllabic Mandarin words that varied in the consonants, vowels and syllable structure. During training participants heard a syllable whilst viewing two of the four standard diacritic representations (i.e. →, ↗, ∨, ↘, which are iconic in nature). They were asked to pick out the picture of the arrow that corresponded to the tone. At test, participants chose which tone they had heard out of a choice of all four diacritics. There were also two generalisation tasks, one testing generalisation to untrained items and one testing generalisation to a new speaker. Native American English speakers showed significant improvement in the accuracy of tone identification after eight sessions of HV training over 2 weeks, and this generalised to both new words and a new speaker. In a follow up study, Wang, Jongman & Sereno (2003) used the same training paradigm to test whether learning transferred to production. They recruited participants taking Mandarin courses and asked them to read through a list of 80 Mandarin words written in Pinyin (an alphabetic transcription) before and after training. They found improvements in production, although these were mainly seen in pitch contour rather than pitch height.

These studies suggested that as with segmental phoneme contrasts, HV training may also facilitate the learning of tone contrasts. However, Wang et al. (1999) and Wang, Jongman & Sereno (2003) did not directly contrast high and low variability training materials. Perrachione et al. (2011) investigated this contrast directly. They trained native American English speakers with no previous knowledge of Mandarin (or any other tonal language), using English monosyllabic pseudowords combined with Mandarin tones 1, 2, and 4 (→, ↗ & ↘). The training task used either LV (one speaker) or HV (four speaker) input. During the training, participants matched the sound they heard with one of three pictures of concrete objects presented, where the three words associated with these pictures were minimal trios that differed only in tone. Participants were tested on their ability to generalise their learning to new speakers. Importantly, Perrachione et al. (2011) were also interested in the role of individual differences in learning. Therefore, they also determined participants’ baseline ability to perceive the tone contrasts prior to training using a Pitch Contour Perception Test. In this task, participants heard a vowel produced with either Mandarin tone 1, 2 or 4 whilst viewing pictures of standard diacritics associated with these tones (→, ↗ & ↘), and were asked to select the arrow that corresponded to the tone. Based on performance in this task, the researchers grouped participants into high and low aptitude groups. The results showed that whilst the LV group outperformed the HV group during training (presumably due to accommodation to a repeated speaker throughout the task), there were no differences between the high and LV groups during test. Critically however, there was an interaction between an individuals’ aptitude categorization and the type of variability training: Only participants with high aptitude benefitted from HV training, while those with low aptitude actually benefitted more from LV training. It is important to note that this interaction was seen in a task which relied on participants’ ability to generalise their learning¹ of tones to an untrained speaker. That is, in a task where we would expect that exposure to multiple speakers would be beneficial since it should allow learners to better dissociate the tones from the particular speakers used in training. These results, therefore, suggest that only the high aptitude learners can take advantage of this benefit. Another training study by Sadakata & McQueen (2014) also explored the relationship between input variability and individual aptitude in lexical tone training, though using different training and testing materials. They trained native Dutch speakers (with no prior knowledge of Mandarin or any other tonal language) using naturally produced bisyllabic Mandarin pseudowords. The two syllables in each word either had Tone 2 followed by Tone 1, or Tone 3 followed by Tone 1, and each tone pair was randomly assigned one of two numeric labels (e.g. for one participant Tone 2-Tone 1 was labelled ‘1’, Tone 3-Tone 1 was labelled ‘2’). During the training task, participants identified the tone pair type of each stimulus by choosing the correct numeric label (e.g. hear /pasa/ with Tone 2-Tone 1, correct response is 1). Thus, in contrast to the study by Perrachione et al. (2011), participants did not need to learn the meaning of each word. Input variability was manipulated, with three levels (low/medium/high). In contrast to the work by Perrachione et al. where the HV and LV conditions differed only in terms of the number of speakers, in this study variability was increased both by including more speakers and more items. Specifically, the number of different vowels used in the bi-syllabic sequences was manipulated: the LV group encountered only one vowel (e.g. pasa, casa, lasa, etc.) whereas the medium and HV groups encountered four different vowels (pasa, pesa, pisa, pusa; casa, cesa, cisa, cusa; lasa, lesa, lisa, lusa etc.). Participants were tested on the trained items (i.e. using trained speakers and trained items). Generalisation was also examined in a number of ways by looking at (1) trained items spoken by an untrained talker; (2) pseudowords containing untrained vowels (3) pseudowords in which the order of tones in the bi-syllables were reversed (i.e. a novel position), and (4) items where the tone was embedded in a sentence context. As in the study by Perrachione et al. (2011), Sadakata & McQueen (2014) also tested individual aptitude but with a different method. They employed a categorization task using stimuli from a six step Tone 2 to Tone 3 continuum (created using natural productions of the two tones with the Mandarin vowel /a/ as endpoints and linearly interpolating between these endpoints). Participants were asked to identify if the sound they heard was more like Tone 2 or Tone 3, and a categorization slope was obtained for each participant providing a measure of their ability to discriminate this contrast, which is generally found to be the most challenging tone contrast for L2 learners of Mandarin. Participants were grouped according to their slopes, and this grouping was entered as a factor in the analyses of tests of learning, along with the effect of training condition (high-medium-low) and the interaction between factors. For the test with trained speakers and items, there was no group level effect of variability condition, however there was an interaction between variability and aptitude similar to that reported by Perrachione et al.: Participants with high aptitude benefitted from HV training, while those with lower aptitude benefitted more from LV training. For the generalisation tests, participants showed above chance performance in all but the new position condition, demonstrating an ability to generalise their learning of tone across different dimensions. However, they did not demonstrate an overall benefit of higher variability in any of the transfer tests, nor, did variability interaction with aptitude. Note that the overall lack of a HV benefit is again surprising, particularly for test items with untrained talkers and novel items, since the manipulations in training should specifically work to increase generalisation along these dimensions.

In sum, the two studies which have directly compared high and low variability input in training Mandarin tone contrasts have not found the predicted benefit of HV on generalisation, either when varying just speakers or when varying speakers and items. However, both of these studies found an interaction between participant aptitude and variability condition. The results of these studies thus provide mutually corroborating evidence—using somewhat different training and testing methods—that the ability to learn from HV input is dependent on learner aptitude, although it should be noted that this interaction was found in a task with untrained speakers in one study (Perrachione et al., 2011), but in a task with trained stimuli in the other (Sadakata & McQueen, 2014).

Why might the ability to benefit from varied training materials depend on participant aptitude? Perrachione et al. (2011) suggest that one reason why low aptitude participants may struggle with multi-speaker input is that the speakers were intermixed during training: This requires trial-by-trial adaptation to each speaker, which was not required in the corresponding single speaker LV conditions. This may place a burden on learners (see Mattys & Wiget, 2011; Nusbaum & Morin, 1992, for evidence that intermixed multi-speaker stimuli are difficult even for L1 processing and that this interacts with constraints on working memory and attention). To test this, Perrachione et al. (2011) conducted a second experiment in which items from each speaker were presented in separate blocks (as is more common in HVPT). This improved performance during the training task compared with unblocked training for low aptitude learners only, confirming the hypothesis that switching between speakers on a trial-by-trial basis during training interferes with learning for low aptitude learners. On the other hand, Sadakata & McQueen (2014) employed a training paradigm in which speakers were blocked in the HV condition, yet they still found the interaction with aptitude. However, recall that in their experiment they also manipulated item variability, yet only speakers were blocked by session, not items. Thus, it remains possible that trial-by-trial inconsistency at the level of items could explain some of the greater difficulty of low aptitude learners in their study.

The current study

The fact that neither of the tone training studies found an overall benefit of high over LV in tone generalisation is surprising in light of the phonetic literature and the predictions of the computational model (Apfelbaum & McMurray, 2011; Ramscar & Baayen, 2013) mentioned above. Moreover, as the previous authors point out, if it is actually the case that learning from multiple voices is more or less effective for different groups of learners, this has important implications for the design of L2 training tools. For this to be the case, it is important to establish the generalisability of the findings to different contexts and materials, particularly those which are relevant in an L2 learning context. We suggest that what L2 learners are most interested in developing is their ability to use tone when mapping a word’s phonological form to its meaning (and vice versa). In this light, the paradigm used by Sadakata & McQueen (2014) lacks ecological validity in looking only at mapping to abstract tone categories. On the other hand, Perrachione et al. (2011) do train form-meaning mappings, yet, unlike Sadakata & McQueen (2014) they use English pseudo-word stimuli, which has the consequence that learners do not simultaneously have to deal with non-native segments and tones, as in a real world L2 learning situation. Furthermore, although there is limited data on the differences between words and non-words in production, it has been noted that non-words may have different properties from real words even within the same language (Scarborough, 2012) and may be more clearly articulated (Hay, Drager & Thomas, 2013; Maxwell et al., 2015). Thus, using non-words might make stimuli slightly easier to learn than if real words were used.

The current training study addresses these issues in a partial replication of the previous work: We use stimuli produced by native Mandarin speakers which are real words in that language. This design choice follows earlier studies such as Wang et al. (1999) using a paradigm in which participants are trained to identify word meaning on the basis of tone. In contrast to the previous studies, we also trained the contrasts between all four tones (six tone contrasts) rather than just three (on the assumption that learners are interested in learning the complete set of contrasts within a particular language). We note that these design choices potentially increase the difficulty of our training materials compared to previous work. A key question was whether these choices would impact the interaction between learner aptitude and the benefits of more variable training materials.

We followed Perrachione et al. (2011) in varying variability along one dimension only—speaker variability, keeping training items identical across conditions. We also followed Perrachione et al. (2011) in comparing HV input which was blocked by speaker, with input that was not, making three training conditions: LV (one speaker), HV (four speakers intermixed within each training session) and blocked training (four speakers each presented in separate blocks). Note that our choice to manipulate only talker-variability means that the HV blocked condition is matched to the LV condition in terms of trial-by-trial inconsistency, unlike in Sadakata & McQueen (2014) where, even though they blocked by speaker, the HV condition contained more trial-by-trial variability in terms of items. We predicted that the difficulty of HV input for lower aptitude participants would be greater in the unblocked condition, thus potentially increasing the likelihood of seeing the predicted interaction between variability and learner aptitude. On the other hand, blocked input is more usual of HVPT (Iverson, Hazan & Bannister, 2005; Logan, Lively & Pisoni, 1991) and may increase the possibility of seeing an overall benefit of speaker variability on generalisation.

We used two perceptual tasks designed to tap individual aptitude. These were adapted from those used in Perrachione et al. (2011) and Sadakata & McQueen (2014). However, while the previous studies grouped participants into one of two categories (high aptitude vs low aptitude) based on the aptitude score, in the current study they were used as continuous measures. This allowed us to avoid assigning an arbitrary ‘cut off’ for high vs low aptitude groups, and the loss of information which occurs when an underlying continuous variable is turned into a binary measure. Note that the statistical approach used in the current paper (logistic mixed effect models) allowed us to include continuous predictors and look at their interactions with other factors.

A further extension in the current study is that we use several new outcome measures to test learning and generalisation. First, most similar to the task used in Perrachione et al. (2011) was a picture identification task which was a version of the training task (2AFC picture identification) without feedback. Following Perrachione et al. (2011) we included untrained-speaker items, where benefits of speaker variability in training should be most apparent. However, bearing in mind that Sadakata & McQueen (2014) actually found the key interaction with aptitude only in the test with trained stimuli, we also included trained-speaker test items.

We also included a second perceptual task which did not involve knowing specific form-meaning mappings and thus had the benefit that it could be conducted both pre- and post-test. This was a three interval oddity task which required participants to pick the odd-one-out after hearing three words spoken aloud, each by a different speaker. Two of the tokens were productions of the same word and the third differed only in the tone (e.g. bā, Tone 1; bā, Tone 1; bà, Tone 4). Because all three tokens are physically different, it requires the listener to focus on the phonological level ignoring irrelevant acoustic differences. Furthermore, the use of three speakers forces the listener to ignore irrelevant speaker-specific differences, making it especially challenging (Strange & Shafer, 2008). This task used untrained speakers in every trial, so that every test-item required generalisation to new speakers² . In addition, here it was possible to use both trained and untrained items. Note that even though the variability over items is matched across conditions, it is possible that varying speaker specific cues might also thus promote generalisation across this dimension. If this is the case, a HV benefit may be stronger for untrained items than trained items.

Finally, we also tested production using a picture naming task at post-test, in which participants were required to name the pictures used in training in Mandarin. We also conducted a word repetition task, which had the benefit that it could also be conducted at pre-test, and that we could use both trained and untrained words (as for the three-interval oddity task discussed above). Although there is evidence HVPT can benefit the production of tones (Wang, Jongman & Sereno, 2003), there has been no direct examination of whether HV training materials are more effective than LV training materials for production. However, more generally in the L2 vocabulary learning literature, training with multiple speakers has been found to lead to better recall in a picture naming task (Barcroft & Sommers, 2005), suggesting that the HVPT advantage should extend to production measures.

In sum, the current experiment assessed whether individuals benefit from high over LV perceptual training when learning novel L2 tone contrasts, and whether this interacts with learner aptitude. We used measures of aptitude taken from previous studies, but a training paradigm with real Mandarin stimuli embedded in a vocabulary learning task, which trained discrimination of all six Mandarin tone contrasts. Learning and generalisation were measured in multiple tests of both perception and production. In general, the current design increased ecological validity and likely also increased the difficulty of the learning task relative to previous work. It is possible that increasing difficulty could exacerbate differences between learners of different aptitudes, potentially increasing the effect. On the other hand, it is also possible that the increased difficulty might make HV input much harder for all participants, decreasing or removing the specific benefit of HVPT for high aptitude learners.

Participants

A total of 60 adults recruited from UCL Psychology Subject Pool participated in the experiment, 20 in each of the three conditions (LV, HV, high variability blocked (HVB)). Participant information is summarized in Table 1. There was no difference between these groups in age, F (2,57) = 1.95, p = 0.15. Participants had no known hearing, speech, or language impairments. Written consent was obtained from participants prior to the first session. Each participant was paid £45 at the end of the study.

All participants except three were native English speakers. Of the remaining three, one participant (LV condition) was a native bilingual of English and Hindi, one participant (HV condition) was a native French speaker, and one participant (HV condition) was a native Finnish speaker. Critically, participants had no prior experience of Mandarin Chinese or any other tonal language. On average, participants had learned 2.4 (SD = 0.8) languages and the average age for starting to learn the first L2 was 12.6 years (SD = 1.3).

Ethical approval was given by the UCL Research Ethics Committee with the approval number 6176/002.

Stimuli

Procedure

The experiment involved three stages (see Fig. 1): Pre-test (session 1), training (sessions 2–7), and post-test (session 8). Participants were required to complete all eight sessions within 2 weeks, with the constraint of one session per day at most. The majority of sessions took place in a quiet, soundproof testing room in Chandler House, UCL. The remaining sessions took place in a quiet room in a student house.

Participants were given a brief introduction about the aim of the study and told that they were going to learn some Mandarin tones and words. They were explicitly told that Mandarin has four tones (flat, rising, dipping and falling) and that the tonal differences were used to distinguish meanings. The experiment ran on a Dell Alienware 14R laptop with a 14-inch screen. The experiment software was built using a custom-built software package developed at the University of Rochester.

The specific instructions for each task were displayed on-screen before the task started. After each task, participants had the opportunity to take a 1-min break. The tasks completed in each session are listed in Fig. 1 and described in more detail below. Note that the Pitch Contour Perception Test and Categorisation of Synthesised Tonal Continua were carried out at the beginning of the first session as they provided the measure of individual aptitude prior to exposure to any Mandarin stimuli. There was no time limit for making responses in any of the tasks. Participants wore a pair of HD 201 Sennheiser headphones throughout the experiment with audio stimuli presented at a comfortable listening level.

Training task

Participants completed the training task in Session 2–7. On each trial, participants heard a Mandarin word and selected one of two candidate pictures displayed on the computer screen. The two pictures always belonged to the same minimal pair. Feedback was provided about whether the answer was correct (a green happy face appeared) or incorrect (a red sad face appeared). If the correct choice was made, a picture of a coin also appeared in a box on the left-hand side of the screen, with the aim of motivating participants to try to earn more coins in each subsequent session of training. After that, everything but the correct picture was removed from the screen and the participant heard the correct word again. In the lower right corner of the screen a trial indicator of X/288 was displayed where X indicated the number of trials completed. This tool helped participants to keep track of their performance (see Fig. 2).

There were 18 picture/word pairs used. Each word was used as the target four times. Thus, each picture pair appeared eight times, resulting in 288 trials per session. Participants were assigned to one of the following conditions: LV, HV and HVB (with the assignment of speakers counterbalanced—see Table 3). Each training session lasted for approximately 30 min.

In the LV condition, only one speaker was used. In the HV conditions, four speakers were used. For each participant, each of their six training sessions was identical. In the HV condition without blocking, all of the speakers were heard in each of the training sessions, with the order randomised so that speaker varied from trial to trial. In contrast, in the HV blocked condition, from Day 1 to Day 4 of training (i.e. Session 2–5), only one speaker was involved on each day’s training session, (with the trained speaker that was used in the test tasks (e.g. F1 for Version 1) always occurring on Day 3 (i.e. Session 4)); on Days 5 and 6 of training (i.e. Sessions 6 and 7), participants heard all four speakers, each in a separate block, with each word being repeated twice in each voice on these days. In all three conditions, the order of items was randomised within each session.

Statistical approach

Three different sets of frequentist analyses are reported. First, we conducted the analysis on two individual aptitude measures Categorisation of Synthesised Tonal Continua and Pitch Contour Perception Test. The primary aim of these analyses was to ensure that the three groups did not differ at pre-test, however we also looked for possible differences at post-test. Second, separate analyses are reported on data from the tests administered pre- and post-training (i.e. Word Repetition task and Three Interval Oddity task), the data collected during Training and the data from the two tasks administered only at post-test (i.e. the Picture Identification task and Picture Naming task). These analyses explored the effects of our experimentally manipulated conditions on the various measures of Mandarin tone learning. Third, analyses were conducted exploring the role of aptitude in each of these tasks (section ‘Analyses with Individual Aptitude’). Specifically, we wanted to see whether aptitude interacted with variability-condition in predicting the benefits of training, in line with the predictions of previous research (Perrachione et al., 2011; Sadakata & McQueen, 2014).

Except where stated, analyses used logistic mixed effect models (Baayen, Davidson & Bates, 2008; Jaeger, 2008; Quené & Van den Bergh, 2008) using the package lme4 (Bates et al., 2013) for the R computing environment (R Development Core Team, 2010). Logistic mixed effect models allow binary data to be analysed with logistic models rather than as proportions, as recommended by Jaeger (2008). In each of the analyses, the factor variability-condition has three levels (LV, HV and HVB) which we coded into two contrasts with LV as the baseline (LV vs HV, LV vs HVB). An exception to this is the training data, where a model containing all three conditions would not converge and we took a different approach, as described in the section ‘Training’. We also included the interactions between these contrasts and the other factors. We used centred coding which ensured that other effects were evaluated as averaged over all three levels of variability-condition (rather than the reference level of LV³ ). Similarly, for the Three Interval Oddity task, we included a trial-type factor. The purpose of this was to control for the fact that participants were likely to find some trial types easier than others due to the gender of the speakers producing the stimuli (see section ‘Three Interval Oddity Test (Pre-Post Test)’). We therefore coded a factor trial-type with three levels (neutral, easy, hard–see method) and included contrasts with neutral (‘neutral vs easy’ and ‘neutral vs hard’) using centred coding. In order to perform the analysis comparing pre- and post-test performance, test-session was coded as a factor with two levels (pre-test/post-test) with ‘pre-test’ set as the reference level. This allowed us to look at the (accidental) possible differences between the experimental conditions at the pre-test stage, as well as whether post-test performance differed from this baseline. All other predictors, including both discrete factor codings with two levels (item-novelty in the Word Repetition and Three Interval Oddity tasks, and voice-novelty in the Picture Identification task) and numeric predictors (training-session) in the Training data analyses and the individual difference measures in the models reported in the section ‘Analyses with Individual Aptitude’), were centred (i) to reduce the effects of collinearity between main effects and interactions, and (ii) so that the main effects were evaluated as the average effects over all levels of the other predictors (rather than at a specified reference level for each factor). We automatically put experimentally manipulated variables and all of their interactions into the model, without using model selection (except for trial-type in the Three Interval Oddity task which works as a control factor and for this factor we only used its main effect and the interaction with test-session). However, we did not inspect the models for all main effects and interactions. Instead, we report the statistics which were necessary to look for accidental differences at pre-test, and those related to our hypotheses. We aimed to examine whether the training improved participants’ performance on both untrained items and untrained voices and whether such improvement was modulated by their individual aptitudes. Participant is included as a random effect and a full random slope structure was used (i.e. by-subject slopes for all experimentally manipulated within-subject effects (test-session, voice-novelty, item-novelty) and interactions, as recommended by Barr et al. (2013). In some cases the models did not converge and in those cases correlations between random slopes were removed. Models converged with bound optimization by quadratic approximation (BOBYQA optimization; Powell, 2009). R scripts showing full model details can be found here: https://osf.io/wdh8a/.

In addition to the frequentist analyses, in order to aid interpretation of key null results we also included Bayes factor analyses. Our approach for these is described within the relevant section (Section ‘Bayes Factor Analyses’).

Individual aptitude tasks

The pitch contour perception test

The predicted variable was whether a correct response was given (1/0) on each trial. The predictors were the contrasts between variability-conditions (LV vs HV; LV vs HVB) and test-session (pre-test, post-test). There was no significant difference between the LV and HV groups (β = −0.35, SE = 0.26, z = −1.38, p = 0.17) or between the LV and HVB groups (β = 0.17, SE = 0.26, z = 0.66, p = 0.51) at pre-test on this measure. Participants showed significant improvement after training (β = 0.21, SE = 0.05, z = 4.13, p < 0.001), which can be seen in Fig. 3.

Thus, the three participant groups did not differ in their pre-test performance and the groups showed equivalent improvement from pre- to post-test. Given that this measure is affected by training, we used participants scores at pre-test as our measure of individual differences in the analyses reported in the section ‘Analyses with Individual Aptitude’.

Training

A model containing data from all three conditions did not converge; however two separate models, one including the LV and HV conditions, and the other the LV and HVB conditions (with condition as a factor with two levels), did converge. In each case the predicted variable was whether a correct response was given (1/0) on each trial. The predictors were the numeric factor training-session (1:6) and the factor variability-condition which had two levels (Model 1: LV vs HV; Model 2, LV vs HVB). The mean accuracy is displayed in Fig. 4.

In both models, there was an effect of training-session (Model 1: β = 0.49, SE = 0.04, z = 11.52, p < 0.001; Model 2: β = 0.53, SE = 0.04, z = 12.17, p < 0.001): Participants’ performance increased significantly over time, with additional training sessions. Overall, the LV group performed better than both the HV group (β = −0.79, SE = 0.16, z = −5.03, p < 0.001) and the HVB group (β = −0.83, SE = 0.32, z = −2.61, p < 0.01). However, the LV vs HV contrast was also modulated by an interaction with test-session (β = −0.19, SE = 0.04, z = −4.59, p < 0.001), as was the LV vs HVB contrast (β = −0.35, SE = 0.08 z = −4.33, p < 0.001). From Fig. 4 it can be seen that the LV and the HVB group did not differ in the first session (i.e. where they get identical input) but the difference gradually increased over the next few sessions. For the LV and the HV group, they differed starting from the first session and this difference continued to increase throughout training.

Perceptual tests

Three interval oddity task

The predicted variable was whether a correct response was given (1/0) on each trial. The predictors were test-session (pre-test, post-test), variability-condition (LV vs HV, LV vs HVB), trial-type (neutral vs easy, neutral vs hard) and item-novelty (trained item, untrained item). The mean accuracy is displayed in Fig. 5.

At pre-test, there was no significant difference between the LV and HV groups (β = −0.002, SE = 0.14, z = −0.01, p = 0.99) nor between the LV and HVB groups (β = 0.12, SE = 0.14, z = 0.86, p = 0.39), suggesting that the groups started at a similar level. However, performance with the ‘untrained’ was significantly greater than performance on the ‘trained’ items at pre-test (β = −0.31, SE = 0.06, z = −4.95, p < 0.01), suggesting incidental differences between item sets. As expected, at pre-test participants performed significantly better on ‘easy’ trials (where the target speaker had a different gender) than ‘neutral’ trials (where all three speakers had the same gender, β = 0.40, SE = 0.08, z = 5.09, p < 0.01) and ‘neutral’ trials were marginally easier than ‘hard’ trials (where one of the foil speakers had the odd gender out, β = −0.14, SE = 0.08, z = −1.81, p = 0.07).

Overall, participants’ performance increased significantly after training (Mpre = 0.59, SDpre = 0.21, Mpost = 0.66, SDpost = 0.19, β = 0.31, SE = 0.05, z = 6.54, p < 0.001). The interaction between test-session and item-novelty was not significant (β = 0.14, SE = 0.09, z = 1.49, p = 0.14), suggesting no evidence that training had a greater effect for trained words than for untrained words. Critically, there was no interaction with test-session for either the contrast between the LV vs the HV conditions (β = −0.01, SE = 0.12, z = −0.12, p = 0.90) or the contrast between the LV vs the HVB conditions (β = 0.01, SE = 0.12, z = 0.11, p = 0.91) and they were not qualified by any higher level interactions with item-novelty (LV vs HV: β = −0.1, SE = 0.22, z = −0.64, p = 0.52; LV vs HVB: β = 0.13, SE = 0.22, z = 0.57, p = 0.57). This suggests no evidence that the extent to which participants improved on this task between pre and post-test differed according to variability-conditions, or that this differed for trained vs untrained items.

Although not part of our key predictions, we also looked to see if there was evidence that participants improved more with the easier or harder trials. In fact, the interaction between test-session and the contrast between ‘easy’ and ‘neutral’ was significant (β = −0.27, SE = 0.11, z = −2.39, p = 0.02) while the contrast between ‘neutral’ and ‘hard’ was not (β = 0.12, SE = 0.11, z = 1.06, p = 0.29). This was due to the fact that there was improvement for ‘neutral’ (Mpre = 0.57, SDpre = 0.14, Mpost = 0.65, SDpost = 0.15) and ‘hard’ trials (Mpre = 0.54, SDpre = 0.16, Mpost = 0.65, SDpost = 0.15) but not for ‘easy’ trials (Mpre = 0.66, SDpre = 0.16, Mpost = 0.68, SDpost = 0.15).

Picture identification

The predicted variable was whether a correct response was given (1/0) on each trial. The predictors were the factor voice-novelty (Trained voice, Untrained voice) and the factor variability-condition which had two contrasts (LV vs HV, LV vs HVB). The mean accuracy is displayed in Fig. 6.

There was a main effect of voice-novelty (β = 1.07, SE = 0.16, z = 6.53, p < 0.001) reflecting higher performance in trials with trained voices. Although participants in the LV group performed better than those in the HV group (β = −0.71, SE = 0.32, z = −2.23, p = 0.03), there was no significant difference between the LV and the HVB group (β = −0.14, SE = 0.32, z = −0.44, p = 0.66) and there was a significant interaction between voice-novelty and both the LV-HV contrast (β = −1.19, SE = 0.35, z = −3.43, p < 0.01) and the LV-HVB contrast (β = −1.11, SE = 0.36, z = −3.08, p < 0.01). Breaking this down by variability-condition: for each condition there was significantly better performance with trained than untrained voices (LV: β = 1.83, SE = 0.29, z = 6.42, p < 0.001; HV: β = 0.64, SE = 0.23, z = 2.86, p < 0.01; HVB: β = 0.73, SE = 0.26, z = 2.82, p < 0.01), indicating greater ease with the familiar voice. Breaking down by voice-novelty: For the trained voice, performance was higher in the LV condition than in either the HV or HVB conditions, although this was only significant for the LV vs HV contrast (LV vs HV: β = −1.30, SE = 0.44, z = −2.97, p < 0.01; LV vs HVB: β = −0.70, SE = 0.45, z = −1.55, p = 0.12). Importantly, for untrained voices, neither of the contrasts between conditions was significant (LV vs HV: β = −0.12, SE = 0.26, z = −0.45, p = 0.65; LV vs HVB β = 0.41, SE = 0.27, z = 1.51, p = 0.13), indicating no evidence for greater generalisation following HV training.

Production tests

Word repetition

Tone accuracy

The predicted variable was whether a correct response was given (1/0) on each trial (as identified by the coder). The predictors were test-session (pre-test, post-test), variability-condition (LV vs HV, LV vs HVB) and item-novelty (trained, untrained). The mean accuracy, split by test-session and training condition, is shown in Fig. 7.

At pre-test, there was no significant difference between the LV and the HV group (β = 0.01, SE = 0.18, z = 0.06, p = 0.95) nor between the LV and the HVB group (β = 0.11, SE = 0.18, z = 0.64, p = 0.53), suggesting the groups started at a similar level. There was also no difference between trained and untrained words at pre-test (β = −0.02, SE = 0.07, z = −0.26, p = 0.80).

Across the three groups, participants’ performance increased significantly after training (Mpre = 0.71, SDpre = 0.09, Mpost = 0.79, SDpost = 0.09, β = 0.40, SE = 0.08, z = 5.29, p < 0.001). There was no significant difference in the improvement for trained and untrained items (word-type by test-session interaction: β = 0.13, SE = 0.10, z = 1.22 p = 0.22). Critically, the interactions between the variability contrasts and test-session were not significant (LV vs HV: β = −0.10, SE = 0.18, z = −0.55, p = 0.58; LV vs HVB: β = −0.11, SE = 0.18, z = −0.62, p = 0.54), and they were not qualified by any higher level interactions with item-novelty (LV vs HV: β = 0.15, SE = 0.25, z = 0.61, p = 0.54; LV vs HVB: β = −0.31, SE = 0.26, z = −1.21, p = 0.23). This suggests there is no evidence that participants’ improvement in their production of tones was affected by their variability-condition, or that this differed for trained vs untrained items.

Pinyin accuracy

The predicted variable was whether the participants produced the correct string of phonemes (1/0) in each trial (as determined by Rater 2). The predictors were test-session (pre-test, post-test), variability-condition (LV vs HV, LV vs HVB) and item-novelty (trained, untrained). Mean pinyin accuracy is displayed in Fig. 8.

At pre-test, there was no significant difference between the LV and the HV group (β = −0.01, SE = 0.11, z = −0.11, p = 0.91) nor between the LV and the HVB group (β = −0.03, SE = 0.11, z = −0.24, p = 0.81), suggesting that the groups started at a similar level. However, participants did better on untrained words than trained words at pre-test (β = 0.21, SE = 0.07, z = 3.11, p < 0.01), suggesting potential accidental differences in these items. Participants showed significant improvement after training (Mpre = 0.54, SDpre = 0.09, Mpost = 0.58, SDpost = 0.19, β = 0.15, SE = 0.05, z = 3.38, p < 0.01). However, there was no evidence that different variability conditions resulted in different amounts of improvement (test-session by LV vs HV: β = 0.05, SE = 0.11, z = 0.46, p = 0.65; test-session by LV vs HVB: β = −0.12, SE = 0.11, z = −1.08, p = 0.28) or any interaction between variability condition, test-session and item-novelty (LV vs HV: β = 0.11, SE = 0.22, z = 0.51, p = 0.61; LV vs HVB: β = −0.14, SE = 0.22, z = −0.64, p = 0.52). This suggests there is no evidence that participants’ improvement in pinyin accuracy was affected by their variability-condition, or that this differed for trained vs untrained items.

Picture naming

Tone accuracy

The predicted variable was whether a correct response was given (1/0) on each trial (as identified by the coder). There was only one predictor, variability-condition (LV vs HV, LV vs HVB) for both models. The descriptive statistics are displayed in Fig. 9.

Participants in the LV group showed no significant difference compared with the HV group (β = −0.34 SE = 0.19, z = −1.81, p = 0.07) and the HVB group (β = −0.10, SE = 0.19, z = −0.52, p = 0.61. This suggests there is no evidence that participants’ ability to produce the tones accurately differed according to their variability-condition.

Pinyin accuracy

The predicted variable was whether the participants produced the correct string of phonemes (1/0) in each trial and there was a single predictor variability-condition (LV vs HV, LV vs HVB). For both models there was no significant difference between variability conditions (LV vs HV: β = 0.09, SE = 0.23, z = 0.41, p = 0.68; LV vs HVB: β = 0.12, SE = 0.23, z = 0.51, p = 0.61). This suggests there is no evidence that participants’ pinyin accuracy differed according to their variability-condition.

Analyses with individual aptitude

In order to look at the effect of learner aptitude and the interaction between this factor and variability condition, we first calculated the mean accuracy at pre-test on the Pitch Contour Perception Test for each participant. This score (scaled by a factor of 10, so that each one unit increase in aptitude corresponded to a 10% higher performance in the Pitch Contour Perception test) was centred and used as a continuous predictor (aptitude) and added to each of the models reported above. In addition, we added the interaction between this factor and key experimental factors (see Table 4). Based on Perrachione et al. (2011) and Sadakata & McQueen (2014), for our measures of tone-learning, HV should benefit high aptitude participants only, while LV would benefit low aptitude participants only. In our design, we used a continuous measure of individual ability rather than a binary division of high and LV. We therefore predicted a stronger positive correlation between aptitude and amount of learning in the HV condition than in the LV condition. In the tests administered only post training (i.e. Picture Identification and Picture Naming) this would show up as an interaction between aptitude and condition. In the models for the pre- and post-test data (i.e. Three Interval Oddity and Word Repetition) this would show up as a three-way interaction between condition, test-session and aptitude. We also looked at the interactions between these factors and voice-novelty (Picture Identification) and item-novelty (Three Interval Oddity and Word Repetition). Note that there are no clear directional hypotheses here: Perrachione et al. (2011) found the interaction in a test with untrained voices and trained items, and Sadakata & McQueen (2014) found the interaction in a test with trained voices and trained items. For training, in principal both the two-way interaction of aptitude by condition and the three-way interaction of aptitude by condition by training-session are of interest. However, it was not possible to fit a converging model containing the three-way factor⁴ .

Each model reported in Table 4 contained all the fixed effects included in the original models in addition to the fixed effects listed in the table (note that to avoid convergence issues due to over complex models, we did not attempt to include the complete set of interactions for every combination of experimental variables with aptitude—only those for which we had predictions). We attempted to have full random effects structure for these fixed effects however in some cases we had to remove correlations between slopes due to problems with convergence and for one of the models with the training data we had to remove the random slope for training session). Note that we don’t include models for the pinyin measures, since our measure of aptitude is relevant to tone learning only. For each of the new models we first confirmed that adding in the new effects and interactions with the individual measures did not change any of the previously reported patterns of significance for the experimental effects (see script https://osf.io/wdh8a/) for full models.

The results are shown in Table 4. Aptitude is a positive predictor of performance in each of the tests and in training, with p-values significant or marginal in each case. However there was no interaction between aptitude and any other factor. Thus, there was no evidence that this measure of aptitude correlated with participants ability to benefit from training (no interaction with test-session), nor—critically for our hypothesis—did this differ by training condition (no interaction with condition or with test-session by condition).

Although the analyses use a continuous measure of Pitch Contour Perception Test, for the purposes of visualisation, Fig. 10 (Three Interval Oddity task and Training task), Fig. 11 (Picture Naming and Picture Identification) and Fig. 12 (Word Repetition) use the mean accuracy for participants split into aptitude groups using a median split based on their Pitch Contour Perception Test score.

In sum, participants with higher aptitude measures were better at the tasks, but there is no evidence either that this affected their improvement due to training, or, critically, their ability to benefit from the different variability exposure sets.

Bayes factor analyses

In the analyses reported above, we did not find evidence—in any of our tests—for either of two key hypotheses: (1) the hypothesis that training with multiple speakers leads to greater generalisation to new speakers than training with a single speaker or (2) the hypothesis that there is an interaction between the variability of the training materials and participant aptitude, such that higher aptitude participants benefit more from training with multiple speakers while lower aptitude participants benefit more from training with a single speaker. However, there is a difficulty in interpreting these null results since a non-significant result (p > 0.05) does not tell us whether we have evidence for the null, as opposed to no evidence for any conclusion at all, or even evidence against the null. Thus, we should not reduce our confidence in either of our hypotheses on the basis of the null results reported above (despite the fact that reducing confidence in a theory following non-significant results is common practice)—see Dienes (2014) for discussion. An alternative statistic is a Bayes Factor, which are used to assess the strength of evidence for one theory (H1) over another (the null hypothesis). We therefore supplemented the analyses above by computing Bayes factors for contrasts relating to these two key hypotheses. These are reported in the sections ‘H1: Greater generalization—to Novel Voices and in Production—in the Multiple Speaker Conditions (HV and HVB) than in the LV Condition’ and H1: There is an Interaction Between an Individual’s Tone-Aptitude and Variability-Condition, Such That Participants with Greater Tone-Aptitude Show Greater Performance Following the Multiple Speaker Conditions (HV and HVB) and Those with Lesser Tone Aptitude Show Greater Performance in the Single Speaker Condition (LV)’ below.

Tests of individual aptitude

In the current work, we conducted two tests with the purpose of capturing individual aptitude: The Pitch Contrast Perception Test (following Perrachione et al., 2011) and the Categorisation of Synthesised Tonal Continua (following Sadakata & McQueen, 2014). Although our goal was to measure participants’ baseline aptitude, the tests were conducted both at pre- and post-test, following Sadakata and McQueen, who did not find differences from pre- to post-tests with their categorisation measure, and who used combined data from pre- and post-test to compute participants slopes. Unfortunately, the performance of our own participants suggested that the Categorisation of Synthesised Tonal Continua test was not a good test of aptitude, with the majority of participants failing to meet the slope threshold used in Sadakata and McQueen, and most being unable to consistently categorise the end points of the continua. It is unclear why our results differ from the previous study (we aimed to follow their procedures), but this meant that we were unable to use this as an aptitude measure in our later analyses. The scores on the Pitch Contrast Perception Test alone therefore served as our measure of individual aptitude. Interestingly, preliminary analyses (section ‘The Pitch Contour Perception Test’) demonstrated that performance in this test improved from pre- to post-training. This suggests that this measure is not a ‘pure’ measure of individual differences since it also appears to be affected by experience. Given this, we only used participants’ scores on this test from pre-test as the measure of aptitude in subsequent analyses.

Performance in training

The training task employed in this study was a 2AFC task, where participants had to identify the correct meaning of a Mandarin word based on its tone. The results from training indicate that participants performed better in the single speaker LV training than in either the multiple speaker HV or HVB groups. This difference was present from the first session for the LV-HV contrast, and from the second session for the LV-HVB contrast (i.e. the first session where the two conditions differ), and increased over time for both contrasts. Greater difficulty with multiple speaker input is in line with the findings of Perrachione et al. (2011), although the differences did not emerge so rapidly in that study, possibly due to there being fewer trials per session. Intuitively, repeated exposure to the single speaker in the LV condition allows for greater adaptation to speaker specific cues, whereas in the HV conditions participants have to adapt to multiple speakers. This is particularly difficult in the unblocked HV condition, where trial-by-trial adaptation is needed, which is effortful for participants (Magnuson & Nusbaum, 2007). Importantly, however, for all three groups, their performance gradually increased over each session. In combination with the fact that their performance on the other tasks increased after training, this indicates that the training task and materials were effective. We also explored the role of learner aptitude in this task (as measured by performance on the Pitch Contour Perception Test at pre-test) and whether this influenced participant’s performance differently in the different variability conditions. Overall, aptitude was found to be a significant predictor of performance during training. However, there was no evidence for an interaction with training condition, although our Bayes Factor Analyses suggests that the data here are inconclusive. We return to this finding in the section ‘The Role of High Variability Materials in Training and Generalisation’ below.

Perception tests

We included two perceptual tasks which tapped learning and generalisation due to training: A Picture Identification task administered at post-test and a Three Interval Oddity task administered at both pre- and post-test. The Picture Identification task was a version of the training task without feedback, and is the most similar to the tests used by Perrachione et al. (2011), and Sadakata & McQueen (2014). We used this test to look at learning of the trained stimuli, comparing trained and untrained voices. The three interval oddity task had not been used in the previous studies, but allowed us to use a pre-/post-test design, and also to look at participants’ performance with untrained items. These tests provided evidence that participants improved in their perception of tones following training: They were above chance in using the tone to identify the correct picture in the picture identification task at post-test, and they improved in their ability to discriminate tones in the three interval oddity task (59% performance prior to training, 66% post training). There was also evidence of generalisation across both voices and items: Participants were above chance in identifying the correct pictures even with an untrained voice (although they did show significantly weaker performance than with the trained voice) and they improved in their ability to discriminate the between minimal pair items in the three interval oddity task, even for untrained items.

Our key questions concerned the role of variability in training. First, we were interested in whether there was evidence that exposure to multiple voices during training led to greater ability to generalise across voices at test—that is, greater performance with novel voices in the HV conditions than in the LV condition. We did not see this. In fact, the only effect of variability in this data was a low variability benefit, which we saw in the Picture Identification task for the trained-voice items (seen in the contrast between LV and HV conditions). This mirrors what we saw in training and reflects the greater exposure to this particular speaker in the LV training. However, in the tests tapping generalisation to a novel speaker—that is, in untrained voice trials in the Picture Identification task, and with all of the test-items in the Three Interval Oddity task, there was no difference between variability-training conditions. Bayes factor analyses indicate that in both cases, there was substantial evidence for the null.

The second hypothesis was that there would be an interaction between learner aptitude (as measured by the Pitch Contour Perception Test at pre-test) and variability training condition, such that high aptitude participants would benefit more from HV training. Note that previous work had found this interaction both in tests involving generalisation (Perrachione et al., 2011) and with trained items (Sadakata & McQueen, 2014) so we considered both in our analyses here. There was no evidence of such an interaction in either the Picture Identification or Three Interval Oddity tasks. However, Bayes Factor analyses suggest that the data are inconclusive. We return to these points in the section ‘The Role of High Variability Materials in Training and Generalisation’ below.

Another finding from the Three Interval oddity test that is worth noting, although it did not concern our hypotheses, is that some trial types were harder than others. Recall that this test involved participants hearing three different stimuli each produced by a different speaker, which makes noting the similarity across two of the stimuli much harder—something we discovered in pilot work, where even before training participants were near ceiling with an equivalent task in which the same speaker produced all three stimuli within a single trial. However, analyses of trial-type demonstrated that participants were additionally affected by the gender of the three speakers producing each of the stimuli. Specifically, at pre-test, participants showed best performance for trials where one of the speakers was male and the other two were female, and the target ‘odd man’ was the male speaker (‘easy’ trials). In contrast, they showed worst performance if there was one male and two female speakers, but the ‘odd man’ was one of the female speakers (‘hard’ trials). Middle level performance was shown for trials where all three speakers were female (‘neutral’ trials). This is presumably due to participants relying on perceptual cues associated with speaker gender to do the task. Interestingly, our analyses showed that performance only increased for the trials where the odd one was not the lone male (the ‘neutral’ and ‘hard’ ones), but not for those where the male was the odd man. Given that participants are not near ceiling at pre-test (67%), it is perhaps surprising that their trained knowledge of the tone contrasts does not boost their performance. One possibility is although they are now better able to use tone cues, they are also less likely to use gender based cues, which they may now realise are less reliable, masking improvement based on tone for these particular test items.

Production tasks

In this study, we used two production tasks, a word repetition task administered pre and post training, in which participants repeated back Mandarin words, and a Picture Naming task testing vocabulary recall, which was administered at post-test only. HV perceptual training for tones has been previously found to transfer to production (Bradlow & Pisoni, 1999; Zeromskaite, 2014), however the benefits of HV and LV training have not been contrasted.

In the Word Repetition task, there was a significant, though relatively modest improvement in participants’ ability to reproduce the tone of the stimuli, such that it could be identified by a native speaker (from pre- to post-test: 70–76%) and in the Picture Naming task, participants showed an ability to recall and reproduce the correct tone, although unsurprisingly with less accuracy than in the repetition task (50%). For Word Repetition, we were also able to look at transfer to untrained words: As in the perception tasks, there was once again equivalent improvement for both trained and untrained items. Together, these results provide evidence that purely perceptual training on tone contrast can transfer to production, as well as to novel items.

In addition to looking at the production of tones, we also looked at participants’ ability to produce the correct segmental phonology (pinyin-score). Participants showed a small but significant improvement on this measure in Word Repetition (54% correct at pre-test, 58% at post-test), and some ability to recall the segments in the Picture Naming test (50% correct). This indicates some learning of segmental phonology due to training, despite the fact that the focus of the training task was on training tonal information through the presentation of tonal minimal-pairs.

Turning to the role of variability, the predicted benefit of HV training was not evident in any of the measures in either of the production tasks, with Bayes factor analyses indicating substantial evidence for the null except for the Word Repetition pinyin-measure, where the evidence was ambiguous. With regard to aptitude, although performance on the Pitch Contour Perception Test at pre-test was predictive of participants’ ability to produce tones in both tasks (indicating a relationship between participants perceptual and production ability), we did not find the predicted interaction between aptitude and variability condition in either task. Here however, Bayes Factor analyses suggests that the results are inconclusive. We return to these points about variability below.

The role of high variability materials in training and generalisation

In the current study, across all of the different tests, we did not find either an overall benefit of exposure to HV training materials for generalisation, or any interaction between such a benefit and individual aptitude.

We consider first the lack of overall variability benefit for generalisation. Importantly, in addition to finding a pattern of null results (i.e. p < 0.05) in the frequentist analyses, additional Bayes Factor analyses also found substantial evidence for the null (BF < 0.33) in all but one of the test measures (Word Repetition, Pinyin, where BF = 0.421). Thus, there is good evidence that, at least for these training and test materials, exposure to stimuli from multiple speakers does not lead to greater generalisation in either perception or production. This finding is consistent with the lack of a main effect of variability condition in the transfer tasks in either Sadakata & McQueen (2014) or Perrachione et al. (2011) (see also footnote 1). However it is at odds with other phonetic training studies focused on segmental contrasts (Clopper & Pisoni, 2004; Logan, Lively & Pisoni, 1991; Lively, Logan & Pisoni, 1993; Sadakata & McQueen, 2013) and with the literature demonstrating a HV benefit in vocabulary learning (Barcroft & Sommers, 2005, 2014; Sommers & Barcroft, 2007, 2011). This suggests that this overall variability benefit may be restricted to segmental rather than tonal phonetic learning, at least for speakers of a non-tonal L1.

It is difficult to reconcile the lack of benefit for vocabulary learning in the picture naming task, given the findings of Barcroft & Sommers (2005, 2014) and Sommers & Barcroft (2007, 2011), since this test is quite similar to that used in their experiments. However, one possibility is that this is due to differences in our training set up (i.e. focused on training tonal contrasts) compared with the earlier vocabulary studies. Nonetheless it remains unclear why tone learning should be different from other types of phonetic learning in terms of benefiting from talker-variability. Theoretically speaking, in a framework where all cues compete, variation in idiosyncratic speaker-specific cues would be expected to provide key evidence as to which cues are irrelevant to the phonetic contrast in question (Apfelbaum & McMurray, 2011; Ramscar & Baayen, 2013; Ramscar et al., 2010). This raises the question of how participants in our LV condition are able to generalise at all—that is, how can they identify the phonetically relevant cues compared with the idiosyncratic cues associated with the single speaker to which they were exposed? One possibility is that other variation in our materials aided generalisation, in particular in our real word stimuli, each tone-contrast is encountered with multiple consonants and vowels. If item variability also aids generalisation to new speakers, this might explain why we found equivalent generalisation across conditions instead of seeing greater generalisation in the HV conditions (i.e. even the LV condition is really a HV condition, because of the item variability). On the other hand, Sadakata & McQueen (2014) also saw generalisation even for their LV condition, and in their study this condition lacked variation in terms of both speakers and phonetic contexts. This suggests that the relevant cues for the tone contrasts may be sufficiently acoustically salient for learners to identify them, even when exposure occurs in limited contexts.

Another possibility—and the one suggested by the findings of Sadakata & McQueen (2014) and Perrachione et al. (2011)—is that benefits of HV for generalisation are masked by individual differences. In their studies, only high aptitude participants showed a HV benefit, while low aptitude participants did not. It is possible that for lower aptitude participants, the benefits of exposure to varying, idiosyncratic cues are offset by the greater difficulty that these participants have in attuning to the different speakers during training, as discussed above (section ‘Tests of Individual Aptitude’). This explanation is supported by evidence from a study by Goldinger, Pisoni & Logan (1991) who explored the effect of increasing the processing cost of multi-speaker input in the context of word recall (in the L1). Specifically, they exposed participants to single vs multi-speaker word lists, manipulating presentations rates. They found that single-speaker lists produced better word recall than multiple-speaker lists at short inter-word intervals (less than 2,000 ms) whereas this effect was reversed for longer inter-word intervals. This suggests that increasing encoding difficulty can remove the benefits of multi-speaker exposure. Relatedly, Sinkeviciute et al. (2019) found that young learners have greater difficulty processing multi-speaker training materials in L2 vocabulary learning, and subsequently fail to show a speaker-variability benefit at test. One interpretation of these findings is that age-related capacity limitations may constrain the ability to benefit from speaker variability, supporting the notion that differences in capacity limitations can affect an individual’s ability to benefit from multi-talker training.

Returning to the current study, we did not find an interaction between variability-training and learner aptitude. However, it is important to acknowledge the results of our Bayes factor analyses, which did not find substantial evidence in support of the null over H1 (or H1 over H0) for any of the test tasks. This means that we cannot draw conclusions about this hypothesis from the current data. In theory, we could continue collecting data until we had substantial evidence for either H0 or H1. To explore the feasibility of this, we conducted supplementary analyses to estimate the sample size that might be needed to see substantial evidence for the null (based on the assumption that the error term would reduce in proportion to √SE). Taking the Picture Identification test (the test most similar to previous studies) our results suggests that it would require N > 300—that is, over five times our current sample size. This suggests that this experimental paradigm is not sufficiently sensitive to address this hypothesis.

Given the ambiguity of our findings with regard to the interaction, it is not appropriate to extensively interpret why we do not find the interaction while the previous studies did. However, we note that there are a variety of differences across the studies which could underpin the different findings, if it holds true. For example, the test of individual differences which we use is harder than that used by Sadakata & McQueen (2014) since it uses all six Mandarin vowels (whereas the original study used five, without /y/) and all of the Mandarin tones (where Perrachione et al. used three, without Tone 3). This change also means that that we cannot easily contrast the range of participant scores in the two studies and it may be that the spread of ability of our participant is different from theirs. In addition, our training task is potentially harder than both of the previous studies, that is, involving all four tones in the context of natural Mandarin stimuli in the context of a word learning tasks. Finally, we also note that our statistical analyses are different from both of the previous studies in that they took their continuous aptitude measures and turned these into binary factors using a ‘cut off’, whereas our statistical approach allows us to use them as continuous variables. However, this should in principle make our approach more powerful than in previous studies.

Future directions

If the interaction between aptitude and training condition reported in Sadakata & McQueen (2014) and Perrachione et al. (2011) is to have implications for educational materials, it is important to establish whether it extends to other more naturalistic materials. Given the relatively small samples in these original studies, and the increasing recognition that psychology experiments have been routinely underpowered (Maxwell, Lau & Howard, 2015; and see Vasishth et al. (2018) for a recent demonstration in the area of reading) and that can lead to increases in both type 1 and type 2 error, we suggest that it would be useful to implement a direct, high powered replication of these previous studies. We note that having a sufficient sample to provide substantial evidence for H1/H0 using Bayesian methods, or to obtain 90% power for frequentist methods, would likely require a much larger sample than is standard in these types of studies. Given the time-consuming nature of these multiple session training studies, moving to online testing may be necessary to make this feasible (see Xie et al., 2018 for an example of an acoustic training study done over the web), or alternately multi-lab collaboration may be necessary. Note that this would also allow us to see whether the fact that Perrachione et al., (2011) found their interaction with untrained voices, whereas Sadakata & McQueen (2014) found it only for trained voices, is a true difference (due to the different paradigms) or due to power. Critically, successful replication would allow us to then extend the paradigms in such a way as to explore the factors above. For example, would increasing the number of tones to use all four Mandarin tones and/or using natural Mandarin stimuli affect the interaction between variability in the input and learner aptitude?

Although direct replication will play a useful role in establishing these effects, we believe that ultimately it will also be important to develop a more nuanced approach to measuring the factors leading to different levels of aptitude both in tone learning, and in other types of phonetic learning. We note that here in addition to not seeing the predicted interaction with variability, we also didn’t see interactions between aptitude and training session in any of our tasks, suggesting that our aptitude measure predicted baseline performance on the task and not the ability to improve due to training. In addition, the tasks used to measure ‘aptitude’ are quite similar in nature to the training and test tasks, decreasing their explanatory value. Our ongoing work explores the combined predictive value of a range of measures including measures of attention, working memory and musical ability. Identifying factors which are predictive of aptitude for tone learning has clear implications for teaching and the personalisation of teaching methods.