Reader Comments

Post a new comment on this article

bayes

Posted by AsaWilks on 26 Feb 2009 at 07:09 GMT

i like the idea of applying bayesian reasoning to scientific research. despite doing a lot of econ i get frustrated with econometricians being completely focused on the s.e.'s of one or two slope coefficients. do some multilevel/heirarchical models to understand whats going on in the data first!

someone could argue that priors might make true success rates better not worse our arbitrary thresholds for "statistical significance." but making the error rate in your sim higher than .05 is probably right not just to make the model work well but also because of methodological errors in data gathering and analysis.

maybe im wrong, but one thing i dont see in your simulation model is researchers' systematic desire to shoot down counterintuitive published research, and the tendency of journals to publish articles that do just that. with increasing intensity as the research being shot down gets more "informative". if you included this, i bet you would find that it corrects for some of the inefficiency over the long term.

seeing as we're all open source and everything, here is a cool R graphic about publication bias: http://addictedtor.free.f...