Transcranial magnetic stimulation enhances the specificity of multiple sclerosis diagnostic criteria: a critical narrative review

Search strategy

The search was planned by the entire study team and performed by a single author (NJS).

A single author (NJS) searched the PubMED, Embase, Web of Science, and Scopus electronic databases for studies published between January 1, 1985 (the first year of TMS publication) (Barker, Jalinous & Freeston, 1985) and February 28, 2022. The following search terms were adapted for each database:

(“multiple sclerosis” (all fields) OR “clinically isolated syndrome” (all fields)) AND (“transcranial magnetic stimulation” (all fields)) AND (sensitiv* (all fields) OR specific* (all fields) OR “predictive value” (all fields) OR “likelihood ratio” (all fields) OR “odds ratio” (all fields) OR “risk ratio” (all fields) OR “hazard ratio” (all fields)).

A single author (NJS) also scanned reference lists of relevant review articles and full-text articles.

Study screening

English, peer-reviewed journal articles of original studies were screened by a single author (NJS). Screening criteria were planned by the entire study team using the PICOS format (Schardt et al., 2007).

Population. Adults with MS; diagnosed using validated criteria; and sample size n > 40, to enhance sensitivity and specificity estimates (Langer-Gould et al., 2006; Schaffler et al., 2011).

Intervention. Observational research that reported sensitivity and/or specificity.

Control. Healthy controls, free of neurologic or other disease; persons with alternative diagnoses.

Outcome. Sensitivity and/or specificity of any upper and/or lower extremity TMS technique.

Study. Cross-sectional or case-control studies comparing MS to control participants; or cohort studies following participants from symptom onset to diagnosis.

Data extraction

The approach to data extraction was planned by the entire study team and performed by a single author (NJS). The study team verified all transcribed data.

From study methods, a single author (NJS) transcribed study and participant characteristics, TMS methods, and criteria to determine abnormal TMS results relative to controls (Chipchase et al., 2012; Langer-Gould et al., 2006; Schaffler et al., 2011; Snow et al., 2019).

From study results, a single author (NJS) transcribed 2 × 2 contingency findings (Adeniyi et al., 2016; Glas et al., 2003; McInnes et al., 2018) (Table 2). Sensitivity was considered the percentage of participants with a diagnosis of MS, who exhibited abnormal TMS results (Sensitivity = True Positives ÷ (True Positives + False Negatives)). Specificity was considered the percentage of control participants (i.e., without a diagnosis of MS), who exhibited normal TMS results (Specificity = True Negatives ÷ (False Positives + True Negatives)). All studies provided sufficient information to determine sensitivity. In cases where insufficient information was available to determine specificity, only sensitivity was reported. Sensitivity or specificity estimates below 50% indicated that the TMS outcome performed worse than chance at ruling out or in MS, respectively.

When both sensitivity and specificity outcomes were available, the diagnostic odds ratio (DOR) was estimated (Glas et al., 2003). DOR was calculated as DOR = (True Positives ÷ False Negatives) ÷ (False Positives ÷ True Negatives) (Glas et al., 2003). Any DOR above 1.0 was associated with increased diagnostic accuracy (i.e., an increased odds that an abnormal TMS result was associated with diagnosis of MS) (Glas et al., 2003). DOR values were interpreted as trivial if < 1.68, small if 1.68–3.46, medium if 3.47–6.71, and large if > 6.71 (Chen, Cohen & Chen, 2010). When possible, 95% confidence intervals for sensitivity, specificity, and DOR were estimated using the methods outlined by Glas et al. (2003).

Critical appraisal

The approach to critical appraisal was planned by the entire study team and completed by a single author (NJS). The study team verified all critical appraisal findings.

Risk of bias. Risk of bias was assessed using the QUADAS-2 tool (Whiting et al., 2011), which evaluates studies’ reporting in domains of participant selection, the index test (TMS), the reference standard (MS diagnostic criteria), and participant flow and timing.

Participant selection questions evaluated reporting of the participant selection process and the level of detail used to describe participant samples:

1. Was a consecutive or random sample of patients enrolled?

2. Was a case-control design avoided?

3. Did the study avoid inappropriate exclusions?

Index test questions assessed reporting of TMS data collection, analysis, interpretation, and summarization:

1. Were the index test results interpreted without knowledge of the results of the reference standard (i.e., was blinding employed)?

2. If a threshold was used, was it pre-specified?

Reference standard questions examined reporting of MS diagnostic criteria:

1. Is the reference standard likely to correctly classify the target condition?

2. Were the reference standard results interpreted without knowledge of the results of the index test (i.e., was blinding employed)?

Flow and timing questions appraised reporting of participant exclusions and the timing between MS diagnosis and TMS testing:

1. Was there an appropriate interval between index test and reference standard?

2. Did all patients receive a reference standard?

3. Did all patients receive the same reference standard?

4. Were all patients included in the analysis?

A single author (NJS) answered signaling questions as Yes/No/Unclear, to derive High/Low/Unclear risk of bias for each domain. Based on the risk of bias from each domain, an overall risk of bias rating was assigned to each study (Sterne et al., 2019).

Biomarker validity. To explore whether studies provided sufficient evidence to justify TMS techniques as biomarkers for MS diagnosis, Bielekova & Martin’s (2004) MS biomarker criteria were used. The Bielekova & Martin (2004) criteria classify biomarker studies according to MS-related pathophysiologic process, grade studies’ methodologic quality, evaluate studies’ clinical utility, and assess studies’ clinical usefulness.

MS-specific pathophysiologic processes were classified as:

1. Biomarkers reflecting alteration of the immune system,

2. Biomarkers of blood-brain barrier (BBB) disruption,

3. Biomarkers of demyelination,

4. Biomarkers of oxidative stress and excitotoxicity,

5. Biomarkers of axonal/neuronal damage,

6. Biomarkers of gliosis, and/or

7. Biomarkers of remyelination and repair.

Methodologic quality of studies was based on the following questions:

1. Are complete (raw) data provided?

2. Was there an independent comparison to a reference standard or age- and sex-matched reference group?

3. Was an appropriate spectrum of patients included (e.g., clinical subtypes, sample size)?

4. Were the methods used valid (e.g., data collection, processing, and analysis)?

5. Was there a processing and/or work-up bias (e.g., blinded processing and analysis)?

Clinical utility was evaluated according to the following criteria:

1. Biological rationale (i.e., rational association with a pathogenic aspect of MS).

2. Clinical relevance (i.e., positioned in the causal chain of pathological events leading to a meaningful clinical endpoint).

3. Practicality (i.e., invasiveness of collection, need for serial analyses, reproducibility, ease, cost).

4. Correlation with disease activity (i.e., relationship with clinical [relapses, progression, disability scale] or neuroimaging [lesion numbers, atrophy] end points).

5. Correlation with disability/prognosis (i.e., relationship with disability accumulation over time).

6. Correlation with treatment effect (omitted).

Clinical usefulness was assessed against the following criteria:

1. Sensitivity/specificity (i.e., sensitivity/specificity relative to reference standard).

2. Reliability (i.e., consistency of a measurement across time or raters, probability of false-positive or false-negative results).

3. Evaluation of a biomarker in epidemiological studies or natural history cohorts (i.e., establishing a statistical relationship between the biomarker and clinical endpoint in cross-sectional or longitudinal studies).

4. Evaluation of a biomarker in proof-of-principle clinical trials (omitted).

A single author (NJS) applied ratings of Yes/No/Unclear to each criterion, to derive Yes/No/Unclear ratings for each domain. Based on ratings in each domain, an overall validity rating (Yes/No/Unclear) was assigned to each study (Sterne et al., 2019).

Visual presentation of TMS outcomes

To visually compare the diagnostic accuracy of TMS techniques across studies, a single author (NJS) prepared a Forest plot of DOR point estimates and their 95% confidence intervals, organized by TMS outcome and study. DOR values were coded according to risk of bias rating. DOR values were interpreted as above (Chen, Cohen & Chen, 2010; Glas et al., 2003).

Study characteristics

Study design characteristics are summarized in Table S1. Only two studies prospectively followed participants from symptom presentation to diagnosis (Beer, Rösler & Hess, 1995; Ravnborg et al., 1992) and most used retrospective, case-control designs. No study followed participants with relapsing MS (RMS) to onset of secondary-progressive MS (SPMS); however, four studies cross-sectionally compared RMS to primary-progressive MS (PPMS) or SPMS (Facchetti et al., 1997; Leocani et al., 2006; Schmierer et al., 2002; Tataroglu et al., 2003). One study followed persons with active vs inactive RMS (Caramia et al., 2004). A total of 10 studies tested for subclinical lesions in participants without clinical motor findings (Beer, Rösler & Hess, 1995; Cruz-Martínez et al., 2000; Hess et al., 1987; Jung et al., 2006; Kale et al., 2009; Kandler et al., 1991; Magistris et al., 1999; Mayr et al., 1991; Ravnborg et al., 1992; Tataroglu et al., 2003). Two studies included comparison groups with diseases other than MS (Beer, Rösler & Hess, 1995; Magistris et al., 1999).

Participant characteristics

MS participant characteristics are detailed in Table 3, while control group characteristics are summarized in Table S2. Across all studies, there were 1408 MS participants (median n = 79, range = 44–162) and 690 control participants (median n = 34, range = 10–155). Few studies matched MS and comparison groups for age or sex. Most studies used the Poser criteria to diagnose MS (Poser et al., 1983), whereas no study employed the 2017 McDonald criteria (Thompson et al., 2018). Average disease or symptom duration ranged from 1–10 years (median 4.6 years). Median disability score, measured using the Expanded Disability Status Scale (EDSS) (Kurtzke, 1983) was 2.5 (range 1.5–6).

TMS findings

Table 4 and Fig. 3 summarize diagnostic accuracy findings of all TMS techniques studied. Table S3 highlights TMS methods and Table S4 provides an in-depth summary of TMS findings. Every study reported the sensitivity of the TMS techniques employed. Specificity and DOR could be gleaned from only eight studies (Beer, Rösler & Hess, 1995; Cruz-Martínez et al., 2000; Hess et al., 1987; Magistris et al., 1999; Mayr et al., 1991; Ravnborg et al., 1992; Schmierer et al., 2000; Tataroglu et al., 2003). Only central motor conduction time (CMCT) and MEP size (amplitude or area) were investigated by at least half of the studies reviewed. Therefore, only these techniques are discussed in detail below.

CMCT. Diagnostic accuracy findings for CMCT are summarized in Table 4 and Fig. 3. CMCT was the most frequently studied TMS technique, in 16 studies (Beer, Rösler & Hess, 1995; Caramia et al., 2004; Cruz-Martínez et al., 2000; Facchetti et al., 1997; Hess et al., 1987; Jung et al., 2006; Kale et al., 2009; Kale, Agaoglu & Tanik, 2010; Kandler et al., 1991; Leocani et al., 2006; Magistris et al., 1999; Mayr et al., 1991; Ravnborg et al., 1992; Schmierer et al., 2002, 2000; Tataroglu et al., 2003). For RMS, sensitivity was greatest when upper and lower limbs were combined (median 76%, range 68–83%). Lower extremity CMCT was most specific for RMS (median 100%, range 100–100%), and was associated with the greatest odds of diagnosis (median DOR 39.17, range 22.15–135.00).

CMCT had poor sensitivity for subclinical lesions (median 19%, range 6–59%), but was up to 96% specific. CMCT tended to be more sensitive for both PPMS (median 72%, range 37–92%) and SPMS (median 96%, range 93–100%) than RMS (median 50%, range 30–63%) in head-to-head comparisons. In participants with RMS, CMCT had negligible sensitivity for detecting the onset of new disease activity (Caramia et al., 2004), but normalization of CMCT was 75% sensitive for the recovery from active to inactive disease (Cruz-Martínez et al., 2000).

MEP size. Diagnostic accuracy findings for MEPs are summarized in Table 4 and Fig. 3. MEP size (amplitude or area) was the second-most frequently studied TMS technique, in nine studies (Cruz-Martínez et al., 2000; Hess et al., 1987; Kale et al., 2009; Kale, Agaoglu & Tanik, 2010; Kandler et al., 1991; Mayr et al., 1991; Ravnborg et al., 1992; Schmierer et al., 2000; Tataroglu et al., 2003). Like CMCT, combined findings from upper and lower extremities yielded the highest sensitivity for RMS (median 58%, range 50–66%). When examined separately, the upper and lower extremities were most specific for RMS (median 100%, range 100–100%), compared to combined upper and lower extremities (median 90%, range 86–94%). However, DOR was greatest for combined upper and lower extremities (median 16.78, range 6.00–27.55). MEP size was poorly sensitive for subclinical lesions (median 29%, range 15–67%) and was not examined in relation to PPMS or SPMS, nor in active vs inactive RMS.

Associations between TMS and disease-related outcomes. Table 4 summarizes associations between TMS techniques and disease-related outcomes. Both CMCT and MEP were correlated with cerebellar function (Cruz-Martínez et al., 2000; Hess et al., 1987; Leocani et al., 2006; Tataroglu et al., 2003). CMCT and MEP size were associated with pyramidal function in some studies (Cruz-Martínez et al., 2000; Hess et al., 1987; Jung et al., 2006; Kandler et al., 1991; Leocani et al., 2006; Magistris et al., 1999; Mayr et al., 1991), but not others (Facchetti et al., 1997; Mayr et al., 1991; Ravnborg et al., 1992). TMS outcomes were not related to other functional systems (i.e., brainstem, sensory, visual, cerebral, bowel-bladder) (Hess et al., 1987; Tataroglu et al., 2003). EDSS was significantly associated with both CMCT and MEP size (Cruz-Martínez et al., 2000; Facchetti et al., 1997; Kale et al., 2009; Kale, Agaoglu & Tanik, 2010; Leocani et al., 2006; Pisa et al., 2020; Schmierer et al., 2002). Lastly, CMCT and MEP size were modestly associated with corpus callosum atrophy, total lesion burden, and corticospinal tract lesions some studies (Cruz-Martínez et al., 2000; Kale, Agaoglu & Tanik, 2010; Ravnborg et al., 1992), but not others (Facchetti et al., 1997; Schmierer et al., 2000).

Risk of bias

See Table 5 for a condensed summary of overall risk of bias findings and Table S5 for more detailed results. Only two studies had low risk of bias (Beer, Rösler & Hess, 1995; Ravnborg et al., 1992), whereas the remainder had either unclear or high risk of bias. The principal source of bias was participant selection, resulting in high risk of bias in 12 studies (Facchetti et al., 1997; Hess et al., 1987; Jung et al., 2006; Kale et al., 2009; Kale, Agaoglu & Tanik, 2010; Kandler et al., 1991; Leocani et al., 2006; Mayr et al., 1991; Pisa et al., 2020; Schmierer et al., 2002, 2000; Tataroglu et al., 2003), 10 of which did not avoid a case-control design (Facchetti et al., 1997; Hess et al., 1987; Jung et al., 2006; Kale et al., 2009; Kale, Agaoglu & Tanik, 2010; Kandler et al., 1991; Mayr et al., 1991; Pisa et al., 2020; Schmierer et al., 2000; Tataroglu et al., 2003).

Biomarker assessment

See Table 6 for a condensed summary of biomarker assessment findings and Table S6 for more detailed results. No study demonstrated sufficient validity for TMS biomarker use (Bielekova & Martin, 2004). Specifically, no study supported TMS use in epidemiologic research. Nearly all studies both failed to provide raw or participant-level data, and failed to either establish or adequately report sensitivity and/or specificity (Beer, Rösler & Hess, 1995; Caramia et al., 2004; Cruz-Martínez et al., 2000; Facchetti et al., 1997; Hess et al., 1987; Jung et al., 2006; Kale et al., 2009; Kale, Agaoglu & Tanik, 2010; Kandler et al., 1991; Leocani et al., 2006; Magistris et al., 1999; Mayr et al., 1991; Pisa et al., 2020; Ravnborg et al., 1992; Schmierer et al., 2002, 2000; Tataroglu et al., 2003). Twelve studies did not address the reliability of TMS techniques (Beer, Rösler & Hess, 1995; Cruz-Martínez et al., 2000; Facchetti et al., 1997; Jung et al., 2006; Kale et al., 2009; Kale, Agaoglu & Tanik, 2010; Kandler et al., 1991; Mayr et al., 1991; Pisa et al., 2020; Ravnborg et al., 1992; Schmierer et al., 2000; Tataroglu et al., 2003). However, despite the above shortcomings, nearly all studies justified disease process-specific biological rationale for TMS to investigate MS, namely the ability to characterize demyelination and/or axonal damage.

Diagnostic performance of TMS techniques

Recent reviews have discussed the role of TMS as a biomarker in MS (Alsharidah et al., 2022; Simpson & Macdonell, 2015; Snow et al., 2019; Ziemann et al., 2011). While there is optimism for using TMS to diagnose, monitor natural history, or assess treatment response in MS (Alsharidah et al., 2022), there is also hesitancy towards widespread clinical use of TMS due to lack of sufficient evidence and high risk of bias (Simpson & Macdonell, 2015; Snow et al., 2019). In the current review, studies failed to justify the validity of TMS techniques as biomarkers for MS diagnosis; however, some outcomes could help characterize corticospinal conduction loss throughout the disease course.

TMS elicits corticospinal motor responses partly by stimulating axon terminals or axonal bends in superficial presynaptic layer II/III/V myelinated neurons at the gyral crown of the precentral gyrus (Siebner et al., 2022). Given MS is characterized by attacks of inflammatory demyelination and axonal damage (in the context of ongoing axonal degeneration) (Pachner, 2021; Reich, Lucchinetti & Calabresi, 2018), these processes can intuitively be characterized using TMS measures of CNS conduction (Vucic et al., 2023; Ziemann et al., 2011). For example, CMCT, MEPs, and triple stimulation technique (TST) could theoretically serve this role (Alsharidah et al., 2022; Simpson & Macdonell, 2015; Snow et al., 2019; Vucic et al., 2023; Ziemann et al., 2011). In the present review, only CMCT and MEP size had sufficient evidence to evaluate their diagnostic performance.

Only CMCT was sensitive for MS (median 75%), when combining findings from upper plus lower extremities. Both CMCT and MEP amplitude had high specificity for MS (median 100%). DOR was greatest and most consistent for CMCT of the lower extremities (median DOR 25.15). Both techniques were generally correlated with pyramidal function, cerebellar function, and EDSS; however, their associations with MRI findings (lesions, atrophy) were inconsistent. Both techniques had poor sensitivity for subclinical lesions, whereas CMCT was 96% specific for subclinical disease. CMCT was 75% sensitive for recovery from active disease but had negligible sensitivity for new disease activity. Lastly, CMCT was highly sensitive for SPMS and PPMS.

Overall, CMCT had the greatest diagnostic performance of the TMS outcomes reviewed (Fig. 3). Two of the eight CMCT studies had low risk of bias (Beer, Rösler & Hess, 1995; Ravnborg et al., 1992), two had unclear risk of bias (Cruz-Martínez et al., 2000; Magistris et al., 1999), and four had high risk of bias (Hess et al., 1987; Mayr et al., 1991; Schmierer et al., 2000; Tataroglu et al., 2003). Risk of bias was primarily attributable to inappropriate exclusions and using case-control designs. No study could demonstrate biomarker validity, due to not providing detailed data, demonstrating poor sensitivity, or failing to justify use in epidemiologic studies. However, in the high-quality studies, estimated DOR was 6–14, suggesting abnormal CMCT was associated with a large, 6 to 14 times, increased odds of MS (Fig. 3) (Chen, Cohen & Chen, 2010).

A major strength of the 2017 McDonald criteria (Thompson et al., 2018) is its high sensitivity, owed to using MRI for evidence of lesion dissemination in space and time in persons following a clinical demyelinating episode. In one study, the sensitivity of the 2017 McDonald criteria was as high as 100% (Gobbin et al., 2019). CMCT therefore has little additive value to enhance the sensitivity of MS diagnostic criteria. However, the greatest strength of CMCT would be its high specificity (median 100%), where the 2017 McDonald criteria was as low as 14% specific in one study (Gobbin et al., 2019). CMCT could reduce false-positive diagnoses (Schwenkenbecher et al., 2019), by “ruling-in” persons with corticospinal conduction loss who were identified as having MS as per the 2017 McDonald criteria.

Alternative utilities for TMS techniques

The current review also found a compelling role for TMS in characterizing MS natural history. For example, CMCT was highly sensitivity for both progressive MS and higher disability status. Given most of the evidence herein is cross-sectional, the authors cannot assign any causal relationship between corticospinal conduction deficits and disease progression or disability accumulation (Pachner, 2021). Nevertheless, CMCT could identify early neurodegeneration to help diagnose a transition to SPMS or revise the diagnosis of RMS to PPMS. The potential role of CMCT in identifying occult neurodegeneration would be especially important, given there are limited treatment options for progressive MS and the diagnosis of PPMS or conversion of RMS to SPMS requires evidence of chronic and irreversible disability accumulation (Hamdy et al., 2022; Thompson et al., 2018). To better establish whether CMCT could expedite the diagnosis of progressive MS subtypes will require more evidence from prospective longitudinal studies.

Limitations

Despite the novelty of this review in terms of addressing the diagnostic performance of TMS and critically appraising the TMS biomarker literature, several limitations should be noted. First, the 2017 McDonald criteria (Thompson et al., 2018) currently represents the gold standard of MS diagnosis. None of the articles reviewed used these criteria, and most used the Poser criteria (Poser et al., 1983). While the authors could not identify any past research that directly compared Poser and 2017 McDonald criteria, successive iterations of the McDonald criteria have generally been shown to diagnose MS earlier and more frequently than the Poser criteria (i.e., enhanced sensitivity) (Brownlee et al., 2015), but with compromised specificity and a higher rate of misdiagnosis (Tintoré et al., 2003). Sensitivity and specificity of the Poser criteria have been estimated at 87% and 94%, respectively (Engell, 1988; Izquierdo et al., 1985), whereas sensitivity and specificity of the 2017 McDonald criteria range between 68–100% and 14–61%, respectively (Filippi et al., 2018, 2022; Gobbin et al., 2019; van der Vuurst de Vries et al., 2018). In past work, the addition of CMCT did not enhance the sensitivity of the Poser criteria to increase MS diagnoses, but increased specificity and reduced MS misdiagnoses (Beer, Rösler & Hess, 1995). It presently is unclear how TMS techniques would perform in the context of the 2017 McDonald criteria, and this question should be addressed in future research.

Next, this critical review is based on only 17 studies of a select few TMS techniques. Methods like paired-pulse and dual-coil TMS, or TMS-EEG, offer unique ways to explore intracortical excitability in excitatory, inhibitory, and neuromodulatory interneurons; other regions important to sensorimotor function (e.g., premotor cortex, supplementary motor area, cerebellum, somatosensory cortices); and distant non-motor regions (Rossini et al., 2015). Such techniques could characterize biologically plausible disease mechanisms not reviewed here, such as acute excitotoxicity or chronic neurodegeneration, linked to disease activity and progression, respectively (Chaves et al., 2019; Snow et al., 2019).

Lastly, given this is a critical narrative review, the results must be interpreted more carefully compared to a systematic review or meta-analysis. The studies included in the review are heterogeneous in terms of study design, sample size, participant characteristics, and TMS methods. The current approach to data synthesis and interpretation does not take study heterogeneity into account. Moreover, because the authors did not produce a single estimate for the diagnostic performance of each TMS technique, there is greater onus on the reader to interpret the findings. Nonetheless, the authors provide a protocol-driven review, following evidence-based methods for data extraction and critical appraisal.