Reader Comments
Post a new comment on this article
Post Your Discussion Comment
Please follow our guidelines for comments and review our competing interests policy. Comments that do not conform to our guidelines will be promptly removed and the user account disabled. The following must be avoided:
- Remarks that could be interpreted as allegations of misconduct
- Unsupported assertions or statements
- Inflammatory or insulting language
Thank You!
Thank you for taking the time to flag this posting; we review flagged postings on a regular basis.
closePreschoolers' Precision of the Approximate Number System Does Not Predict Later School Mathematics Performance
Posted by GuillaumeRousselet on 20 Sep 2011 at 11:21 GMT
The model was significant, with ANS precision accounting for 28% of the variance in TEMA-3 score, r2 = .278, t (16) = 2.405, p = .030 (Figure 3). This demonstrates an association between ANS precision prior to schooling and mathematics performance after the onset of formal instruction. As an indication of the strength of this association, even the concurrent measure of FSIQ was less predictive of TEMA-3 score than was preschool ANS; when evaluated alone, FSIQ at primary school accounted for approximately 7% of variation in concurrent TEMA-3 performance r2 = .068, p = .312.
http://plosone.org/article/info:doi/10.1371/journal.pone.0023749#article1.body1.sec3.sec2.p2
Figures 3 and 4 report a total of 4 correlations, two of them being significant using a p<0.05 cut-off. After correcting for multiple comparisons, these effects are no longer significant. In Figure 3 left, although the authors report a positive relationship between the two variables, most of the subjects (10 out of 17) actually show a clear negative relationship, and the significant association seem to be due to a cloud of 5 subjects in the top right corner. In keeping with this observation, a robust measure of association flags the 5 data points in the top right corner as outliers, and suggests that there is no evidence in favor of an association between ANS and math performance.
The data were extracted from the figures using the mac software GraphClick version 3.0 (Arizona Software, 2008). Using these extracted data points, we could replicate the authors' original correlation values, and the new analyses suggest that the two significant correlations in Figures 3 and 4 are due to outliers. In the new analyses, we used a skipped correlation, whereby the bivariate outliers are removed and Spearman correlation is applied to the remaining data points with adjustments for pruning the data (Wilcox, Introduction to Robust Estimation and Hypothesis Testing, 2005). Outliers were detected in the bivariate space using a projection technique in conjunction with the MCD (minimum covariance determinant) and a MAD (median absolute deviation) rule.
In sum, correcting for multiple comparisons or using a robust measure of association suggests that the authors’ conclusions are unsubstantiated.
Figure 3 left:
Authors: R2=0.278, r=0.5273, p=0.030
Our analyses: Pearson r=0.5279, p=0.029;
Spearman rs=0.4525, p=0.068;
Skipped correlation rp=-0.473, t=2.08, tcrit=2.73
For the skipped correlation, the observed t value (t) must be as large or larger than the critical t value (tcrit) for the effect to be considered significant assuming a type I error rate of 0.05.
Figure 3 right:
Authors: R2=0.001, r=-0.0316, p=0.886
Our analyses: Pearson r=-0.0382, p=0.884;
Spearman rs=-0.0294, p=0.911;
Skipped correlation rp=0.147, t=0.58, tcrit=2.73
Figure 4 left:
Authors: R2=0.324, r=-0.5692, p=0.017
Our analyses: Pearson r=-0.5706, p=0.017;
Spearman rs=-0.605, p=0.012;
Skipped correlation rp=-0.457, t=1.99, tcrit=2.73
Figure 4 right:
Authors: R2=0.034, r=-0.1844, p=0.479
Our analyses: Pearson r=-0.1870, p=0.472;
Spearman rs=-0.1936, p=0.455;
Skipped correlation rp=-0.0824, t=0.32, tcrit=2.73
Preschoolers' Precision of the Approximate Number System Predicts Later School Mathematics Performance
Michele_M_Mazzocco replied to GuillaumeRousselet on 30 Sep 2011 at 18:58 GMT
We thank the reader for voicing concerns that may be shared by other readers, and appreciate the opportunity to respond.
The reader’s general comments concern the integrity of our data analysis and interpretation. Specifically, he is concerned that we did not correct for multiple comparisons, that we reported two of four correlations significant at the "p"<.05 level, and that we did not consider outliers in our data analysis.
Here we explain that our analyses were appropriately executed and reported with utmost care and transparency.
The reader is correct - we did not adjust alpha; instead, we explicitly reported exact p values for all correlations, allowing readers to consider the practical significance of each. The two correlations that were significant (at levels "p" = .03 and "p" = .017) were both in the predicted direction, as were the two correlations that failed to reach significance ("p"s>.45). (The latter were conducted to test the specificity of the former. ) Hence, all four correlations reported in the figures conformed to our hypotheses. The general tendency was for positive associations among analyses between ANS precision at preschool and mathematics-related tasks at school age, and a lack of such associations when the school-age tasks being examined were non-numerical.
P-values for Correlations are not typically corrected for multiple comparisons as done in post-hoc tests for the Analysis of Variance. Follow-up analyses were conducted with a mathematics factor indicated by TEMA and RAN Numbers residuals and predicted by the ANS. Results indicated a strong positive association ( "Beta"= .78) with a p-value of .010. A verbal factor indicated by RAN Letters and Vocabulary was not tenable.
With regard to the reader’s suggestion that we remove outliers, we followed a principled approach in considering whether to exclude participants, and provided a transparent discussion of these considerations in the Results section of our manuscript. Briefly, in view of the variation in ANS precision reported with larger samples of preschoolers (e.g., Halberda & Feigenson, 2008; Libertus et al., 2011, cited in the manuscript), we would anticipate a wide range of precision levels in our preschool sample; the full range should be represented in our analyses. Moreover, our preliminary analysis showed that participants’ performance conformed to Weber’s law, supporting the contention that participants engaged the ANS skills being measured, and providing a solid rationale for inclusion of all participants. Still, we adopted a conservative approach in our second set of analyses by omitting five participants whose responses on the ANS task were at chance. Both sets of analyses are reported in the main paper. The omission of nearly one third of an already small sample significantly affects whether the sample is truly representative of the population of interest, so the primary analyses included all 17 children.
Finally, we believe it best to clearly present such findings as it may serve as a catalyst for future research in this area. No single study is conclusive and we look forward to more studies focused on the associations between early ANS precision and later mathematics skills.
Acknowledgment: We consulted with Dr. Kevin Grimm, Quantitative Psychologist, in preparing this response.