While I commend Shariff and Rhemtuall’s (SR hereafter) research question and endeavor, their results are likely an artifact of multicollinearity, a classic statistical issue that wreaks havoc on parameter estimates and standard errors. To illustrate, sociology of religion scholars who work with the World Values Survey (WVS) know that the survey items SR use to operationalize belief in heaven, hell, and God – “Which, if any, of the following do you believe in?” – are highly correlated. Take, for instance, similar data from the 1999-2004 WVS. I code the belief variables as the proportion of respondents who answer “Yes” to the aforementioned question, which yields a country sample of 54 for belief in heaven, hell, and God, respectively. The correlation matrix for the three variables is as follows: God and heaven (r = .82), God and hell (r = .77), and heaven and hell (r = .93). All three of these correlation coefficients are highly suspect, especially the latter. Furthermore, using modeling techniques similar to SR (i.e., linear regression) with a logged homicide rate for the WVS’s respective survey year (this is the data I had available), the results are even more suspect. First, regressing logged homicide on belief in heaven and belief in hell yields statistically insignificant coefficients in a direction opposite to what SR find: belief in hell (coef. = 2.51, p = .068) and belief in heaven (coef. = -1.37, p = .315). Second, when adding classic controls to this model (i.e., logged GDP per capita and logged Gini coefficient) the coefficients remain statistically insignificant but the directions of effect switch: belief in heaven (coef. = .06, p = .965) and belief in hell (coef. = -.64, p = .638) (Variance Inflation Factors for belief in heaven and belief in hell are 9.52 and 10.62, respectively). Third, when logged homicide is regressed on belief in heaven alone (i.e., log(homicide) = beta1*heaven + error) and logged homicide is regressed on belief in hell alone (i.e., log(homicide) = beta1*hell + error), both coefficients are positive: belief in heaven (coef. = .96, p = .073) and belief in hell (coef. = 1.24, p = .018).

Although the data used here is for one wave of the WVS and for one type of crime (and is largely preliminary), my results suggest that SR’s findings are, in all likelihood, biased and inconsistent and the product of multicollinearity. To dispel my claim, the authors should provide the following information to PLoS ONE comments: (1) a correlation matrix for all independent variables used, (2) models with only belief in heaven and belief in hell predicting all forms of crime, (3) models randomly dividing the sample in two and predicting all forms of crime with only belief in heaven and belief in hell, and (4) variance inflation factors (i.e., VIFs) for all models. Moreover, SR should discuss possible suppressor effects if they think multicollinearity is not an issue but the signs for belief in heaven and belief in hell switch directions when they’re in the same model. And, finally, the authors should provide a justification for the # of control variables and covariates they use when the sample size for each model is noticeably small (this was difficult to determine as the sample size was not provided for each model).

Blaine Robbins
Department of Sociology
University of Washington