Figures
Abstract
Paired associative stimulation (PAS) is a widely used transcranial magnetic stimulation (TMS) paradigm to non-invasively induce synaptic plasticity in the human brain in vivo. Altered PAS-induced plasticity has been demonstrated for several diseases. However, researchers are faced with a high inter- and intra-subject variability of the PAS response. Here, we pooled original data from nine PAS studies from three centers and analyzed the combined dataset of 190 healthy subjects with regard to age dependency, the role of stimulation parameters and the effect of different statistical methods. We observed no main effect of the PAS intervention over all studies (F(2;362) = 0.44; p = 0.644). The rate of subjects showing the expected increase of motor evoked potential (MEP) amplitudes was 53%. The PAS effect differed significantly between studies as shown by a significant interaction effect (F(16;362) = 1.77; p = 0.034) but post-hoc testing did not reveal significant effects after correction for multiple tests. There was a trend toward increased variability of the PAS effect in older subjects. Acquisition parameters differed across studies but without systematically influencing changes in MEP-size. The use of post/baseline quotients systematically indicated stronger PAS effects than post/baseline difference or the logarithm of the post/baseline quotient. The non-significant PAS effects across studies and a wide range of responder rates between studies indicate a high variability of this method. We were thus not able to replicate findings from a previous meta-analysis showing robust effects of PAS. No pattern emerged regarding acquisition parameters that at this point could guide future studies to reduce variability and help increase response rate. For future studies, we propose to report the responder rate and recommend the use of the logarithmized post/baseline quotient for further analyses to better address the possibility that results are driven by few extreme cases.
Citation: Lahr J, Paßmann S, List J, Vach W, Flöel A, Klöppel S (2016) Effects of Different Analysis Strategies on Paired Associative Stimulation. A Pooled Data Analysis from Three Research Labs. PLoS ONE 11(5): e0154880. https://doi.org/10.1371/journal.pone.0154880
Editor: Robert Chen, University of Toronto, CANADA
Received: January 14, 2016; Accepted: April 20, 2016; Published: May 4, 2016
Copyright: © 2016 Lahr et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: This is a pooled analysis across nine studies (see Table 1). Data may be accessed by contacting the corresponding author, as data upload to a public repository is not covered by the individual participant consent.
Funding: The article processing charge was funded by the German Research Foundation (DFG) and the Albert Ludwigs University Freiburg in the funding programme Open Access Publishing.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Neuronal plasticity is the basis of learning and memory and leads to changes on a molecular, cellular and systemic level. On the synaptic level, long-term potentiation (LTP) and depression (LTD) are omnipresent mechanisms of neuronal plasticity. This bidirectional synaptic plasticity can be induced by tetanic stimulation at high (LTP) or low (LTD) frequencies or by associative pre- and postsynaptic stimulation and characteristically depends on the activity of postsynaptic NMDA receptors [1]. LTP/LTD have been studied extensively in animal tissue slices [2], and also in surgically removed human hippocampus specimens [3]. However, it cannot be assessed in the human brain in vivo.
Paired-associative stimulation (PAS) is one of the most frequently used transcranial magnetic stimulation (TMS) protocols to non-invasively induce neural plasticity in the intact human brain [4–6]. For PAS, electrical stimulation of a peripheral nerve (e.g. median nerve) and TMS of the contralateral primary motor cortex (M1) are repetitively coupled. Depending on the exact timing of the stimuli, this leads to an increase or decrease of motor evoked potentials (MEP). At an interstimulus-interval of 25 ms, MEP amplitudes are increased, and at an interstimulus interval of 10 ms, the amplitudes are decreased [5]. MEP amplitude changes have been shown to begin directly after PAS intervention and to last for a duration of at least one hour [4,6].
Experimental paradigms that induce neural plasticity in the intact human brain are often referred to as „LTP-like”as they share some but not all characteristics with LTP and as the underlying mechanisms are not completely understood. For PAS, three requirements for an LTP-like mechanism, namely stimulus-timing dependence [5], NMDA-receptor dependence [7] and associativity [4] have been demonstrated.
A large scale quantitative review provided strong indication for a robust PAS effect which was strongest for an inter-trial interval of 5 or 20 s [6]. On the other hand, a high inter-individual variability and a response to PAS in the expected direction in only 60% and even 39% of participants found in two well powered studies challenge such conclusions [8,9]. Small effect sizes and a high variability have recently been reported not only for PAS but also for other methods of non-invasive brain stimulation and are intensively debated [10,11]. Despite that, systematic alterations of PAS have been demonstrated in a variety of neuropsychiatric disorders, including Alzheimer’s disease [12,13], depression [14], schizophrenia [15] or writer’s cramp [16].
Previous studies identified several factors such as age [8], time of day [17], attention [18] or cortical anatomy [19] to account for some of its variability, but these were not necessarily replicated and even small changes in the parameter sets may impact the results substantially. There is not yet a consensus if optical navigation for TMS coil placement has a positive effect on the variability of MEPs [20–22]. Other factors include the strength of the peripheral electrical stimulation which affects the number of stimulated afferent fibers as well as the strength of the cortical stimulation as this will influence the number of I-waves [23]. One study [8] found that a lower stimulator output necessary to induce a 1mV MEP correlated with a stronger PAS effect.
Finally, the choice of data analysis methods and statistical analyses also influences the results. This concerns the averaging across the individual trials of a participant at a given time point (e.g. at baseline or at various intervals after the PAS-intervention) and the averaging across individuals for each time point but also the transformations (post/baseline differences, quotients, or logarithmized quotients). The majority of studies relies on post/baseline differences for analysis of the main PAS effect by using ANOVA, and on quotients for further analyses such as e.g. correlations.
Our own experience [24–30] and that of others [8,9] with PAS but also relatively well powered studies using other means for non-invasive brain stimulation [19,31] led us to perform a meta-analysis on original data for comparison to the recent review by Wischnewski and colleagues [6]. These authors reported that the PAS protocol remained robust even after removing studies with potential overestimation of effect size. On the other hand, their analysis had to be based on published data while ample evidence indicate that studies with a null effect tend to remain unpublished [32].
We therefore performed a meta-analysis based on our original published and unpublished data from nine studies performed by two TMS research groups at three different laboratories. We aimed at evaluating the robustness of the PAS effect. In addition, we sought to quantify the influence of stimulation parameters, age, gender and statistical methods in the analysis of PAS. We additionally tested for associations between the extend of the PAS effect and age as well as magnetic stimulation strength as both has been indicated in a previous study [8]. Finally, we also examined correlations between PAS and the peripheral electrical stimulation strength.
Methods
Subjects
Healthy control subjects from nine studies conducted at three German centers (Charité Universitätsmedizin Berlin, University Medical Center Münster, University Medical Center Freiburg) were included in this study (Table 1). The studies were approved by the respective local ethics commissions (Ethik-Kommission der Albert-Ludwigs-Universität Freiburg for studies A & B, Ethik-Kommission der Ärztekammer Westfalen-Lippe und der Westfälischen Wilhelms-Universität Münster for studies C & D, Ethikkommission der Charité Universitätsmedizin Berlin for Studies E—I) and all participants gave their written informed consent.
PAS acquisition protocols
The differences of the experimental procedures of the included studies are outlined in Table 2.
Data Processing and Statistical Analysis
The mean MEP size was calculated for each time-point (baseline and post-measurements) and for each subject. Testing MEPs for normality using Shapiro-Wilks test indicated no normal distribution. A repeated measures analysis of variance (rmANOVA) was calculated using SPSS software (Version 22.0) with the factor TIME (three levels: before PAS (pre), directly after PAS (post0) and 15 min after PAS (post15) as these measurements were available in all studies) as repeated measures factor and STUDY (nine levels) as between subjects factor. In case of a significant interaction, Games-Howell correction was applied to post-hoc testing. Degrees of freedom were adjusted by the Huynh-Feldt method, when the assumption of sphericity was violated.
Three different data transformations of the baseline and averaged post MEP measurements were compared: the difference between post and baseline (PASdiff), the post/baseline quotient (PASquot), and the logarithm of the quotient (PASlogquot).
A random-effects regression model was fitted to each of the transformed data sets using the metafor library in R [33] and heterogeneity between studies was assessed with Cochrane’s Q-test. Results are displayed using a Forest plot.
For calculation of the responder-rate, the quotient of the averaged post measurements and the baseline measurement effect was calculated [8]. Subjects attaining values above one were thus considered as PAS-responders. Rank based correlation (Spearman’s rho) between age, TMS intensity (in percent of maximal stimulator output: %MSO), electrical peripheral nerve stimulation intensity (mA), PASlogquot and |PASlogquot| was calculated. Following the approach by Müller Dahlhaus et al. (2008) association between age and the absolute variability (i.e. MEP increases or decreases) induced by PAS was assessed by correlating |PASlogquot| with age using Spearmans’s rho. Association between gender and PASlogquot and between inter-trial interval (5 s or 20 s) and PASlogquot were assessed using point-biserial correlation which is equivalent to a t-test but directly provides a measure of effect size.
To assess the influence of different data transformations, we visualized MEPs relative to baseline from all participants displaying the mean from all participants of a given timepoint and study. This visualization was contrasted with a visualization where the mean and the standard error were calculated on the logarithmized data. To further underline the influence of different averaging methods, we also visualized the mean, the median, and the mean of the log-transformed data that was back transformed to linear space using the following formula:
Results
Data from PAS experiments of nine different studies were analyzed using an rmANOVA on the baseline, post0, and post15 measurements (Tables 1 and 2). There was no main effect of TIME over all studies (F(2;362) = 0.44; p = 0.644; Table 3) while the main effect of STUDY was significant (F(8;181) = 2.04; p = 0.044; Table 3). The overall responder rate was 53.2% (101 out of 190 subjects). The interaction TIME x STUDY was significant (F(16;362) = 1.77; p = 0.034; Table 3) but post-hoc testing did not reveal significant effects after correction for multiple testing (Games-Howell procedure; minimal p-value = 0.256). Responder rates for the individual studies were between 16.7–80% (Table 1).
Results from rmANOVA over all studies (measurements at baseline, post0 and post15). Bold letters indicate significant effects.
None of the three random-effects meta-analyses indicated an overall effect of PAS. The model based on the post/pre quotients indicated a (not significantly) higher effect of PAS (PASquot: 1.11 ± 0.08; S1 Fig), while the models based on the differences (PASdiff: 0.02 ± 0.06 mv; S2 Fig) and logarithmized quotients (PASlogquot: 0.01 ± 0.03; Fig 1) indicated comparable results. Heterogeneity as assessed by Cochrane’s Q was significant at a level p<0.001 for all three models.
The right column lists the corresponding mean and 95% confidence interval for the individual studies, below the estimated effect across all studies is indicated.
There was a trend towards a positive association between variability of PASlogquot and age (rho = 0.13; p = 0.068; Fig 2), and no significant correlation between PASlogquot and age (rho = 0.07; p = 0.308; Fig 2), electrical peripheral nerve stimulation intensity (rho = -0.15; p = 0.091, Fig 3, left panel) and TMS intensity (rho = -0.01; p = 0.897, Fig 3, right panel) or variability of PASlogquot and electrical peripheral nerve stimulation intensity (rho = -0.01; p = 0.139) and TMS intensity (rho = 0.04; p = 0.689). There was also no significant association between PASlogquot and gender (r = 0.01; p = 0.851), the use of neuronavigation (r = 0.01; p = 0.923) or the inter-trial interval during the PAS intervention (5 s vs. 20 s; r = 0.09; p = 0.239).
Letters indicate separate studies (see Table 1).
MSO: maximum stimulator output.
The left panel of Fig 4 displays the PAS effect relative to baseline with means and SEM calculated across all studies indicating relatively high MEP values and a strong effect of outliers. In contrast, data in the right panel were log-transformed before calculating the means and SEM.
Mean MEP amplitudes following PAS are shown relative to the baseline level. Each study is represented by a distinct color as in Figs 2 and 3, each circle represents the measurements of one subject, and each black diamond marker represents the mean of a study at the given time-point. The black line with error bars represents the temporal course of PAS across subjects and studies. The dashed red line represents no change against the baseline measurement. Left panel: normalized data ± SEM. Right panel: same PAS data but log-transformed prior to calculation of means and SEM. Note that the x-axis depicts distinct time points rather than a continuous scale.
The influence of different methods to estimate the average PAS response was further assessed by comparing the mean of the raw data to the median and the back-transformed mean of the logarithmized data. The mean yielded the highest values in 17 out of 19 cases (Fig 5).
Each column of diamond markers represents the different averages of one study: the mean (black), the median (red), and the mean as calculated on the log-transformed data that was back-transformed to linear space (green). The mean values are systematically higher than those of the median or those of the mean calculated on the log-transformed data.
Discussion
Integrating original data from almost 200 subjects, we did not observe a significant effect of PAS (i.e. TIME) but found that the PAS effect differed significantly between studies. However, post-hoc testing did not reveal differences between individual studies after correction for multiple comparisons. The wide range of responder rates together with the differences of the PAS effect between studies argue for a high variability. The results of our study are therefore different from a recent quantitative review showing robust PAS effects across all examined post PAS intervals [6]. Although that study did not access original data from individual subjects, they integrated data from 60 individual studies and made an effort to ensure that results of their meta-analyses were not driven by a number of small scale studies that would report unrealistically large effect-sizes.
Our data were acquired at three sites by two TMS research groups. Although both teams were working independently from each other at the time of data acquisition, we cannot fully exclude the possibility that both teams performed the PAS intervention incorrectly. However, well powered single studies by other research groups with ample experience in neurophysiology [8,9] also found no evidence for a robust PAS effect, and overall low responder rates.
In line with previous findings [8], we found no association between age and the strength of the PAS effect. In contrast to that study, we did not find a smaller but a higher variability of the MEP after PAS with increasing age. Of note, we did not examine a continuous sample across the whole age range and especially middle aged subjects (i.e. 30–50 years old) are underrepresented in the study sample. Between-study differences could therefore influence age effects. There was also no correlation between the strength of the PAS effect and gender, in line with recent work [34]. In additional correlation analyses we examined the effect of the strength of the electrical and magnetic stimulation on the PAS effect and found no significant correlation, in contrast to [8] who reported stronger PAS effects in those requiring a lower stimulator strength to induce a 1 mV MEP.
We were unable to evaluate the effect of different TMS vendors which had recently been suggested [35] as all studies used the same system. In addition, we did not find a systematic difference between studies with and without a navigation system, findings which add to an ongoing and so far inconclusive debate [20–22].
Importantly, we were able to demonstrate in this meta-analysis that the choice of statistical analysis has a distinct impact on the results: MEPs are not normally distributed and, furthermore, are by definition positive, implying that outliers systematically lead to an overestimation of the mean (e.g. single MEP measurements can be above 3 mV, but never below 0 mV). This effect is further magnified when parametric statistics are applied to MEP values relative to a baseline measurement as the baseline measurement takes place directly after adjusting stimulation parameters and coil position to acquire stable MEPs with low variability. Small movements or fluctuations of attention may thus have a higher impact on the subsequent post-measurements. Even if there is no external gold standard to validate the statistical method, the positive skewness of the data leads us to propose to log-transform the post/baseline quotient for further statistical processing as this leads to a more normal data distribution and to use absolute MEP values (baseline- and post-measurements) for a rmANOVA. Although not the focus of this study, the same arguments can also be applied to data from individual trials for a given subject and time point where either mean or median can be used to average across trials and data may already be log-transformed at this stage.
In summary, we demonstrate a high variability of the PAS-protocol leading to an overall non-significant effect of the intervention. Given this high variability, PAS-results in neurological and psychiatric patients should be interpreted with precaution. Conclusions drawn from single subject experiments do not yet seem to be reliable, and studies with higher patient numbers are needed to prove the validity of this paradigm in a clinical context. Detailed description of acquisition parameters, blinding the subject and the examiner to group status and even stimulation protocol (e.g. by involving a second experimenter who switches between excitatory 25 ms and inhibitory 10 ms intervals), carefully controlling for potential confounders such as age, gender, attention and a statistical analysis plan robust against outliers seem the best approach to handle PAS data. Moreover, future should investigate further sources of intra- and intersubject variability as it may have a tractable physiological underpinning. A recent approach to track down the high variability of noninvasive brain stimulation paradigms is using brain-state dependent TMS stimulation and thus adjusting the timing of TMS stimulation by real-time analysis of EEG [36,37]. Brain-state dependent variability of PAS may not only explain discrepancies between PAS studies, but also a high intrasubject variability of PAS [38].
Supporting Information
S1 Fig. Forest plot of PAS-effect calculated with the MEP ratio between the averaged post measurements and baseline (PASquot).
The right column lists the corresponding mean and 95% confidence interval for the individual studies, below the estimated effect across all studies is indicated.
https://doi.org/10.1371/journal.pone.0154880.s001
(TIF)
S2 Fig. Forest plot of PAS-effect calculated with the MEP difference between the averaged post measurements and baseline (PASdiff).
The right column lists the corresponding mean and 95% confidence interval for the individual studies, below the estimated effect across all studies is indicated.
https://doi.org/10.1371/journal.pone.0154880.s002
(TIF)
Acknowledgments
The authors would like to thank Miles Wischnewski for helpful additional comments on his study. The article processing charge was funded by the German Research Foundation (DFG) and the Albert Ludwigs University Freiburg in the funding programme Open Access Publishing.
Author Contributions
Conceived and designed the experiments: J. Lahr AF SK J. List. Performed the experiments: J. Lahr SP AF J. List. Analyzed the data: J. Lahr J. List SP AF WV. Wrote the paper: J. Lahr SP J. List WV AF SK.
References
- 1. Cooke SF, Bliss TVP. Plasticity in the human central nervous system. Brain. 2006;129: 1659–1673. pmid:16672292
- 2. Rioult-Pedotti M-S, Friedman D, Donoghue JP. Learning-Induced LTP in Neocortex. Science. 2000;290: 533–536. pmid:11039938
- 3. Chen WR, Lee S, Kato K, Spencer DD, Shepherd GM, Williamson A. Long-term modifications of synaptic efficacy in the human inferior and middle temporal cortex. Proc Natl Acad Sci. 1996;93: 8011–8015. pmid:8755594
- 4. Stefan K, Kunesch E, Cohen LG, Benecke R, Classen J. Induction of plasticity in the human motor cortex by paired associative stimulation. Brain. 2000;123: 572–584. pmid:10686179
- 5. Wolters A, Schmidt A, Schramm A, Zeller D, Naumann M, Kunesch E, et al. Timing-dependent plasticity in human primary somatosensory cortex. J Physiol. 2005;565: 1039–1052. pmid:15845584
- 6. Wischnewski M, Schutter DJLG. Efficacy and time course of paired associative stimulation in cortical plasticity: Implications for neuropsychiatry. Clin Neurophysiol. 2015;
- 7. Stefan K, Kunesch E, Benecke R, Cohen LG, Classen J. Mechanisms of enhancement of human motor cortex excitability induced by interventional paired associative stimulation. J Physiol. 2002;543: 699–708. pmid:12205201
- 8. Müller-Dahlhaus J, Orekhov Y, Liu Y, Ziemann U. Interindividual variability and age-dependency of motor cortical plasticity induced by paired associative stimulation. Exp Brain Res. 2008;187: 467–475. pmid:18320180
- 9. López-Alonso V, Cheeran B, Río-Rodríguez D, Fernández-del-Olmo M. Inter-individual Variability in Response to Non-invasive Brain Stimulation Paradigms. Brain Stimulat. 2014;
- 10. Ziemann U, Siebner HR. Inter-subject and Inter-session Variability of Plasticity Induction by Non-invasive Brain Stimulation: Boon or Bane? Brain Stimulat. 2015;8: 662–663.
- 11. Hordacre B, Ridding MC, Goldsworthy MR. Response variability to non-invasive brain stimulation protocols. Clin Neurophysiol. 2015;
- 12. Battaglia F, Wang H-Y, Ghilardi MF, Gashi E, Quartarone A, Friedman E, et al. Cortical Plasticity in Alzheimer’s Disease in Humans and Rodents. Biol Psychiatry. 2007;62: 1405–1412. pmid:17651702
- 13. Terranova C, Sant’Angelo A, Morgante F, Rizzo V, Allegra R, Arena MG, et al. Impairment of sensory-motor plasticity in mild Alzheimer’s disease. Brain Stimulat. 2013;6: 62–66.
- 14. Player MJ, Taylor JL, Weickert CS, Alonzo A, Sachdev P, Martin D, et al. Neuroplasticity in Depressed Individuals Compared with Healthy Controls. Neuropsychopharmacology. 2013;38: 2101–2108. pmid:23676792
- 15. Frantseva MV, Fitzgerald PB, Chen R, Möller B, Daigle M, Daskalakis ZJ. Evidence for Impaired Long-Term Potentiation in Schizophrenia and Its Relationship to Motor Skill Leaning. Cereb Cortex. 2008;18: 990–996. pmid:17855721
- 16. Quartarone A, Bagnato S, Rizzo V, Siebner HR, Dattola V, Scalfari A, et al. Abnormal associative plasticity of the human motor cortex in writer’s cramp. Brain J Neurol. 2003;126: 2586–2596.
- 17. Sale MV, Ridding MC, Nordstrom MA. Cortisol Inhibits Neuroplasticity Induction in Human Motor Cortex. J Neurosci. 2008;28: 8285–8293. pmid:18701691
- 18. Stefan K, Wycislo M, Classen J. Modulation of Associative Human Motor Cortical Plasticity by Attention. J Neurophysiol. 2004;92: 66–72. pmid:14724259
- 19. Hamada M, Murase N, Hasan A, Balaratnam M, Rothwell JC. The Role of Interneuron Networks in Driving Human Motor Cortical Plasticity. Cereb Cortex. 2013;23: 1593–1605. pmid:22661405
- 20. Julkunen P, Säisänen L, Danner N, Niskanen E, Hukkanen T, Mervaala E, et al. Comparison of navigated and non-navigated transcranial magnetic stimulation for motor cortex mapping, motor threshold and motor evoked potentials. NeuroImage. 2009;44: 790–795. pmid:18976714
- 21. Jung NH, Delvendahl I, Kuhnke NG, Hauschke D, Stolle S, Mall V. Navigated transcranial magnetic stimulation does not decrease the variability of motor-evoked potentials. Brain Stimulat. 2010;3: 87–94.
- 22. Bashir S, Edwards D, Pascual-Leone A. Neuronavigation increases the physiologic and behavioral effects of low-frequency rTMS of primary motor cortex in healthy subjects. Brain Topogr. 2011;24: 54–64. pmid:21076861
- 23. Di Lazzaro V, Oliviero A, Pilato F, Saturno E, Insola A, Mazzone P, et al. Descending volleys evoked by transcranial magnetic stimulation of the brain in conscious humans: effects of coil shape. Clin Neurophysiol. 2002;113: 114–119. pmid:11801432
- 24. List J, Duning T, Meinzer M, Kürten J, Schirmacher A, Deppe M, et al. Enhanced Rapid-Onset Cortical Plasticity in CADASIL as a Possible Mechanism of Preserved Cognition. Cereb Cortex. 2011;21: 2774–2787. pmid:21531778
- 25. List J, Albers J, Kürten J, Schwindt A, Wilbers E, Flöel A. Reperfusion Does Not Improve Impaired Rapid-Onset Cortical Plasticity in Patients with Severe Stenosis of the Internal Carotid Artery. PLoS ONE. 2012;7: e41004. pmid:22844423
- 26. List J, Duning T, Kürten J, Deppe M, Wilbers E, Flöel A. Cortical plasticity is preserved in nondemented older individuals with severe ischemic small vessel disease. Hum Brain Mapp. 2013;34: 1464–1476. pmid:22331645
- 27. List J, Kübke JC, Lindenberg R, Külzow N, Kerti L, Witte V, et al. Relationship between excitability, plasticity and thickness of the motor cortex in older adults. NeuroImage. 2013;83: 809–816. pmid:23876242
- 28. List J, Hertel-Zens S, Kübke JC, Lesemann A, Schreiber SJ, Flöel A. Cortical Reorganization Due to Impaired Cerebral Autoregulation in Individuals With Occlusive Processes of the Internal Carotid Artery. Brain Stimulat. 2014;7: 381–387.
- 29. Klöppel S, Lauer E, Peter J, Minkova L, Nissen C, Normann CM, et al. LTP-like plasticity in the visual system and in the motor system appear related in young and healthy subjects. Front Hum Neurosci. 2015;9: 506. pmid:26441603
- 30. Lahr J, Peter J, Minkova L, Lauer E, Reis J, Heimbach B, et al. No difference in paired associative stimulation induced cortical neuroplasticity between patients with mild cognitive impairment and elderly controls. Clin Neurophysiol. 2015;0.
- 31. Wiethoff S, Hamada M, Rothwell JC. Variability in Response to Transcranial Direct Current Stimulation of the Motor Cortex. Brain Stimulat. 2014;7: 468–475.
- 32. Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan A- W, Cronin E, et al. Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias. PLoS ONE. 2008;3: e3081. pmid:18769481
- 33. Viechtbauer W. Conducting Meta-Analyses in R with the metafor Package. J Stat Softw. 2010;36: 1–48.
- 34. Strube W, Bunse T, Malchow B, Hasan A, Strube W, Bunse T, et al. Efficacy and Interindividual Variability in Motor-Cortex Plasticity following Anodal tDCS and Paired-Associative Stimulation, Efficacy and Interindividual Variability in Motor-Cortex Plasticity following Anodal tDCS and Paired-Associative Stimulation. Neural Plast Neural Plast. 2015;2015, 2015: e530423.
- 35. Nieminen JO, Koponen LM, Ilmoniemi RJ. Experimental Characterization of the Electric Field Distribution Induced by TMS Devices. Brain Stimulat. 2015;8: 582–589.
- 36. Gharabaghi A, Kraus D, Leão MT, Spüler M, Walter A, Bogdan M, et al. Coupling brain-machine interfaces with cortical stimulation for brain-state dependent stimulation: enhancing motor cortex excitability for neurorehabilitation. Front Hum Neurosci. 2014;8: 122. pmid:24634650
- 37. Sergeeva EG, Henrich-Noack P, Bola M, Sabel BA. Brain-state-dependent non-invasive brain stimulation and functional priming: a hypothesis. Front Hum Neurosci. 2014;8: 899. pmid:25414662
- 38. Fratello F, Veniero D, Curcio G, Ferrara M, Marzano C, Moroni F, et al. Modulation of corticospinal excitability by paired associative stimulation: Reproducibility of effects and intraindividual reliability. Clin Neurophysiol. 2006;117: 2667–2674. pmid:17011821