Jon, you were Reviewer #5 and this was our third round of revisions. It sounds like you did not receive our reply from the PLOS ONE editors, so we provide it below, verbatim.

April 12, 2013

In the response below, we address each question or comment made by the Editor and the Reviewer.

Specific Comments by the Editor and Reviewer:

Editor #1 and Reviewer 5 #1: Reviewer 5 asked to use "the outcome measure for Phase 2 the ERP measure that is determined by the Phase 1 analysis - that is the DIFFERENCE between responses to known versus unknown words, not the absolute value of the response to known words".

Author Reply: The ERP measure used in Phase 2 (i.e., response to known words at the P3 electrode site) was determined a priori based on our hypothesis and the results of Phase 1. Recall that our Phase 1 results revealed significant group level differences between affected children exhibiting less severe social symptoms, affected children exhibiting more severe social symptoms, and TD controls, for known words only—unknown words did not exhibit significant group level differences. Thus the most conceptually simple and straightforward choice of a predictor variable was the response to known words. Examining the predictive power of other ERP components is beyond the scope of this paper, and cannot be done at this time; moreover, it is not good statistical practice in our statistician’s view.

Editor #2 and Reviewer 5 #2: He also wants to see whether the ERP measure better predict the future cognitive and linguistic skills than either cognitive or social measures at time 1. Please find his comment below.

Author Reply: The idea that ADOS Social Total at Time 1 might serve as a predictor of future functional outcome is not included in the Introduction and was not intended to be part of the hypotheses. Reviewers and the previous editor expressed interest in the predictive relationship between ADOS Social Total at Time 1 and future functional outcome, and perhaps we fueled this interest by the initial submission, but it was never our intention to address this question, and we have clarified our theoretical position. We have made small edits in the language used to specify our hypotheses at the end of the Introduction in order to eliminate any potential misunderstanding (see page 7).

As our revisions have indicated, we do not hypothesize that social skills directly predict later linguistic/cognitive development. Nor do we hypothesize/discuss a mediation model to identify/explain a relationship between ADOS Social Total and functional outcomes by including an ERP measure of word processing. In the Introduction and Discussion, we present our theoretical view that the early acquisition of language is tightly coupled to social function. The social brain ‘gates’ the computational mechanisms of early language learning such that the interaction of social factors and language input operate in concert to advance the development of word processing. Our working hypothesis is that our neural indicator is sensitive to the brain’s processing of words and that advancement in this complex learning process predicts later outcomes. To reiterate our previous statement, additional analyses are beyond the scope of the present study.


Editor #3 and Reviewer 5 #3: I agree with the reviewer also in the left-lateralization issue. Since the left-lateralization is reported in the previous literature, the fact that the current study failed to show such an effect should be discussed. Please revise also the parts of the text misleadingly referring to left-lateralization in TD children in the current data.

Author Reply: As Reviewer 5 noted in his comment we received in early January 2013, claims of lateralization require demonstration of significant hemisphere effects. We agreed with Reviewer 5 regarding the use of the word ‘lateralized.’ We also have carefully checked the previous literature, and verified that Mills and colleagues did not report significant hemisphere effects in the differential response to known and unknown words. In other words, left-lateralization is not reported in the previous literature—this impression was introduced by our inappropriate use of the word. The extensive literature cited does not support left hemisphere lateralization in TD children, simply significant differences limited to the left temporal/parietal electrode sites. We revised the manuscript submitted in February 2013 to describe significant effects as ‘limited’ to the left hemisphere, rather than using the word ‘lateralized’, when describing ERP patterns of response to words reported by Mills and colleagues, TD controls in the present study, and affected children with less severe social symptoms in the present study. All three subject groups show a similar pattern of response: None exhibit a ‘lateralized’ response, but all exhibit significant differences limited to a single electrode site in the temporal/parietal region of the left hemisphere. The language used in the revision submitted in February 2013 to describe the pattern of ERP response to words in the studies of Mills and colleagues is also consistent with language used in those publications.


REVIEWER’S COMMENTS:

Reviewer 5, #1: The authors should use as the outcome measure for Phase 2 the ERP measure that is determined by the Phase 1 analysis - that is the DIFFERENCE between responses to known versus unknown words, not the absolute value of the response to known words.

Author Reply: See above reply to Editor #1 and Reviewer 5 #1.


Reviewer 5, #2: Second, the authors really need to show that this ERP measure does a significantly better job of predicting future cognitive and linguistic skills than either cognitive or social measures at time 1. These re-analyses should be very straightforward to conduct and I suspect will not affect the overall conclusions drawn from the paper. Given the ethos of PLoS ONE, the decision of whether or not to publish the paper should be independent of the outcomes of these analyses (I refrained from putting a smiley face here!)

Author Reply: See above reply to Editor #2 and Reviewer 5 #2.


Reviewer 5, #3: (Non) Lateralization of brain responses in TD children: The authors now acknowledge that the differential response is not left-lateralized in the TD children and have replaced "lateralized" with "focal". While this does appear to be an accurate description of the P3 response in autistic children with less severe social symptoms, it is a misleading description of the TD response, which is very clearly bilateral (in fact, the supplementary data suggest that the effect of condition is numerically larger at T4 than for T3. At several points throughout the paper, the authors still write as if the controls showed leftward lateralization. For example:
P15: "Children with ASD exhibiting less severe social symptoms also show a focal response at a single left hemisphere electrode site, consistent with previous research and similar to TD controls in the current study".
P17: "Our Phase 1 results indicate that the pattern of response shown by children with ASD who exhibit more severe social symptoms does not resemble the focal ERP pattern of significant difference between known and unknown words limited to the left temporal/parietal region shown by TD children in this study and in previous work".
Given the extensive literature cited in support of leftward lateralization in control children, there really needs to be some discussion of why the TD children in the current study failed to show such an effect. The authors mention that these effects emerge with development. Is it the case that the TD children tested here are younger than the age at which lateralization typically emerges? If so, then this places the ASD findings in a very different light. In other words, the Less Severe children would be showing precocious lateralization.

Author Reply: See above reply to Editor #3 and Reviewer 5 #3.


Reviewer 5, #4: Choice of ERP predictor for Phase 2: In my previous review, I noted that there were many possible ERP measures that could have been used as predictors. The authors argue, however, that their choice of measure was defined a priori based on the outcomes of Phase 1.

As noted above, the outcome of Phase 1 is that the DIFFERENCE between responses to known versus unknown words at P3 is associated with ADOS Social scores. This comes out in both the median split analysis and in the new correlational analysis.

The response to known words at P3 is not significantly correlated with ADOS social scores, yet this is the measure the authors choose as the predictor.

The authors' justification is that, for known words, but not unknown words, there is an interaction between group and hemisphere. However, this is based on the average of the full set of electrodes (including frontal and central electrodes) and is driven primarily by the fact that participants in the More Severe subgroup show an enhanced response to known words at right fronto-temporal electrodes. In other words, it's very difficult to see how this has any relevance at all to the choice of measure at P3.

If the authors want to convince readers that the choice of measure in Phase 2 was determined a priori (and not merely a target drawn around the arrows) then they absolutely must use the measure that comes out of Phase 1 - ie the difference between responses to known versus unknown words.

Author Reply: See also above reply to Editor #1 and Reviewer 5 #1.
We agree that many variables could have been chosen, but we reiterate that the ERP measure recorded in Phase 1 that serves as the predictor in Phase 2 (i.e., response to known words at the P3 electrode site) was determined a priori based on our hypothesis and the results of Phase 1.

We hypothesized an association between the neural response to speech and classification based on social function: Brain responses in children with ASD exhibiting less severe social symptoms would be similar to TD controls (i.e., significant differences between known and unknown words limited to left temporal/parietal electrode sites), whereas brain responses in children with ASD who have more severe social symptoms would be atypical. We further hypothesized that the defining characteristic of the more typical ERP response to words at the temporal/parietal electrode sites would have implications for future functional outcomes in both subgroups of children with ASD, independent of cognitive function at Time 1. We do show a relationship between ADOS Social Total and the differential response to known and unknown words in Phase 1 Results. However, we provided this data in response to reviewers, and in the context of validating use of the median split, demonstrating that the subgroups of children with ASD do in fact differ in the pattern of ERP response to known and unknown words, even though correlations between the ADOS social total and ERP mean amplitudes to known and unknown words at individual electrode sites were not significant. It is important to note that associations between specific values of the ADOS Social Total and specific characteristics of the ERP response to words are not addressed by our hypotheses. We do not hypothesize that the characteristics of the ERP response most closely associated with ADOS Social Total will have implications for future function. Our hypothesis is that the defining characteristic of the more typical ERP response to words at the temporal/parietal electrode sites would have implications for future functional outcomes.

Phase 1 results revealed significant group level differences only for known words—unknown words did not exhibit significant group level differences. While the significant group level differences for known words are based on the average of the full set of electrodes, the fact remains: one characteristic of the “more typical” response observed in affected children with less severe social symptoms is that it arises from the processing of known words. Planned comparisons reported in Phase 1 results reveal another characteristic of the “more typical” response observed in affected children with less severe social symptoms: a focal response with enhanced negativity to known vs. unknown words that is statistically significant only at a single left parietal electrode site, P3, similar to that of TD controls and unlike the ERP waveforms of affected children exhibiting more severe social symptoms which show a more diffuse right hemisphere response to words significant at two electrode sites. Thus the most conceptually simple and straightforward choice of a defining characteristic of the more typical ERP response to words at the temporal/parietal electrode sites is the ERP response to known words at the P3 electrode site. This choice is well supported by the hypothesis and the full set of Phase 1 results described above. Examining the predictive power of other ERP components is beyond the scope of this paper.


Reviewer 5, #5: Relationship between ADOS Social scores and outcomes: The authors provide clarification of their theoretical position (I'm relieved to find that I understood it correctly the first time). In short, social skills feed into word learning, which has follow-on consequences for later cognitive and linguistic development. The authors do not hypothesize that social skills directly predict later cognitive development. But this is an alternative hypothesis that needs to be ruled out before the authors can make any claims about the mediating influence of word learning skills.

In the revised version, the authors go some way towards addressing this issue, reporting that ADOS Social scores do indeed predict outcome measures 2 years later, although the association is not significant 4 years later. However, this is buried in the results of Phase 1, rather than in Phase 2 where it clearly belongs.

Of course, to convincingly demonstrate that one measure (ERP) is a better predictor than another (ADOS social), the authors would need to perform a direct comparison of the predictive utility of the measures (it is not sufficient to say that one correlation is significant and the other is not). One option would be to determine whether the correlation between the ERP measure and the outcome measure was significantly larger than the correlation between the ADOS Social and the outcome measure. An alternative would be to conduct a partial correlation or stepwise regression analysis to determine whether the ERP measure accounted for variance in outcome measures beyond that accounted for by ADOS Social. My guess is that this will prove to be the case, thereby strengthening the authors' claims.

Author Reply: We do not agree that we need to show that the ERP measure does a significantly better job of predicting future cognitive and linguistic skills than either cognitive or social measures at time 1. This is not our hypothesis: specifically, we do not hypothesize that social skills directly predict later outcome measures, nor do we hypothesize/discuss a mediation model to identify/explain a relationship between ADOS Social Total and functional outcomes by including an ERP measure of word processing. In the Introduction and Discussion, we present our theoretical view that the early acquisition of language is tightly coupled to social function and that the social brain ‘gates’ the computational mechanisms of early language learning such that the interaction of social factors and language input operate in concert to advance the development of word processing. Our working hypothesis is that our neural indicator reveals a critical advancement in the brain’s processing of words and that this advancement predicts later outcomes.

In this context, evaluation of ADOS Social Totals and outcomes as an alternative hypothesis, or evaluation of the relative predictive power of ERP and ADOS Social Total using partial correlations/regression is beyond the scope of the study. Simply reporting the correlations between ADOS Social Totals and outcome measures in the Phase 1 results in the revision submitted in February 2013 already tested the bounds of good statistical practice, since we do not hypothesize relationships between these measures.


Reviewer 5, #6: Abstract: This still refers to "identifying risk", which to most readers would imply "risk of autism". In their response to reviewers are very clear that this is not what they mean - in which case the abstract should specify risk of what!

Author Reply: We agree with the Reviewer, the language is potentially confusing and has been revised: “These findings have theoretical implications and may eventually lead to neural measures that allow early prediction of developmental outcomes as well as more individually tailored clinical interventions, with the potential for greater effectiveness in treating children with ASD” (see page 2).


Reviewer 5, #7: At several points, the authors refer to data in the singular (e.g., "data was?").

Author Reply: All occurrences of “data was” have been replaced with “data were.”


Reviewer 5, #8: P10: I don't understand the sentence "Frequency of words on the individualized lists was highly correlated among the 3 groups."

Author Reply: This sentence has been revised to improve clarity: “Frequency of presentation of specific known and unknown words was similar among the 3 groups (affected children with more severe social symptoms, affected children with less severe social symptoms, and TD controls), with intergroup word frequency correlations ranging from .815 to .903 (p < .0005).” (page 10)


Reviewer 5, #9: P13: When talking about planned comparisons at individual electrode sites, it would be useful to specify which electrodes were planned! If the authors really mean "post-hoc analyses based on the outcome of the ANOVA" then they should say this instead.

Author Reply: Planned comparisons were made at all individual electrode sites measured, consistent with the analyses conducted by Mills and colleagues. The sentence has been revised to read “Data were analyzed in the same manner as previous ERP word studies of TD children [18 - 23] employing repeated measures ANOVA and planned comparisons at all individual electrode sites measured” (page 13).


Reviewer 5, #10: Minor point - I spent quite some time trying to find out what the time window was for analysis. It is in there, but might be worth repeating (e.g when describing the Results or in the figure legend). The authors should also provide a justification for the chosen time window (presumably this is based on previous studies).

Author Reply: The time window has been added to the text in the Results section (page 13). Individual and grand mean ERPs were examined to determine the most appropriate interval consistent with that found to be sensitive to differences in word type by Mills and colleagues, and this information has been added to the Methods section (page 10).


Reviewer 5, #11: P16: "However, there is a significant association between the signature focal negativity for known words at the P3 electrode site, when compared to the overall negativity across all electrode sites, and severity of social symptoms." In their response to reviewers, the authors provide an explanation of how this was calculated. This needs to be included in the paper.

Author Reply: We have added this information as a note (see pages 16 and 27).


Reviewer 5, #12: P17: "the significant group level differences between affected children exhibiting less severe social symptoms and TD controls on one hand, and affected children exhibiting more severe social symptoms on the other, was shown in Phase 1 to be due to the processing of known words". As noted above, this is misleading as it refers to the aggregate of all electrodes, not the specific electrode site chosen.

Author Reply: Phase 1 results revealed significant group level differences only for known words—unknown words did not exhibit significant group level differences. While the significant group level differences for known words are based on the average of the full set of electrodes, the fact is that one characteristic of the ‘more typical’ response observed in affected children with less severe social symptoms is that it arises from the processing of known words. See response to #4 above for full discussion.


Reviewer 5, #13: Composite (a measure of cognitive ability) into linear regression analyses of later receptive language, cognitive ability, and adaptive function." Clarify how the regression was performed. I suspect all predictors entered simultaneously, but this introduces problems of collinearity (ie it doesn't necessarily tell us whether the ERP measure is a better predictor than the Mullens or accounted for unique variance in outcome). As noted above in relation to the ADOS, the strongest test would be a stepwise regression with the ERP measure entered as the final step.

Author Reply: The regression procedure has been clarified in the text (page 19). We first entered Time 1 Mullen Composite (a measure of cognitive ability), and then added Time 1 ERP response as a second step in linear multiple regression analyses of later receptive language, cognitive ability, and adaptive function. Table 5 (page 36) has been revised to designate Model 1 and Model 2 in the analyses, and now provides change in R2 for each model as well as total R2.


Reviewer 5, #14: P19: Regression analysis involving the ADOS Social should be added here.

Author Reply: Evaluation of the relative predictive power of ERP and ADOS Social Total using partial correlations or regression is beyond the scope of the study. See response to Item 5 above.


Reviewer 5, #15: P21: In the context of discussing the very high incidence of non-compliance / unusable data, the authors note that "the signature marker of word processing identified in Phase 1 of the current study successfully predicted outcomes for all [author's emphasis] children with ASD in our sample in Phase 2". I have absolutely no idea what this means.

Author Reply: We agree with the reviewer that this language is confusing. We were trying to say that the prediction of outcomes in children diagnosed with ASD at the age of two years should theoretically be possible in the broader population of children with ASD if obstacles to obtaining usable ERP can be overcome. As you know, not all children produced usable ERP data in our study (as in all studies using ERPs in young children). However, we showed that there were no differences in skill levels for those who did produce usable ERP data and those who did not, suggesting that our results would apply to all children with ASD. This statement is relevant to the notion that the predictor variable may be generalizable. We have revised this section to make it clearer (page 21).



With best regards,
Patricia K. Kuhl, Ph.D.