We thank the three commenters for their questions and criticisms. As the findings of this paper seem to imply both positive and negative aspects to the controversial topic of religion, the paper has generated some impassioned but not always polite, or even printable, responses. As a result, it is both appreciated and refreshing to receive comments of such substance.

We aim to engage each of the commenters’ main points—many of which raise legitimate concerns—while also addressing the overarching question of whether the findings can be trusted to be ‘real.’ At the outset, we should also note to readers that, unlike our original article, this comment—and the data presented within—have not undergone official peer-review.

ISSUES OF MULTICOLLINEARITY: Dr. Robbins is justified in raising concerns about multicollinearity. Given the high correlations between a nation’s rate of belief in heaven and in hell, this was something we considered during our initial data analysis as well. Moreover, the issue came up again during the review process. However, the findings do not actually support the concern.

Multicollinearity arises when two (or more) predictors in a linear regression model are highly correlated. The usual effect of multicollinearity is that the overlapping variance shared by two predictors significantly predicts the dependent variable, making it very difficult for the model to determine what the unique effect of each predictor is – this large degree of uncertainty leads to large standard errors associated with each regression coefficient. In these cases, although each predictor would have been significant on its own, when both are included in the model, often neither are significant. The effect of multicollinearity is that standard errors get bigger (hence “variance inflation factors”) and effects become inconsistent and non-significant.

That is not what is happening with these data. Belief in heaven and belief in hell are indeed highly correlated (r = .92), but it is their unique variance, and not their shared variance, which predicts crime rates. On their own, neither proportions of people who believe in heaven nor hell are significant predictors crime rate. Most of their variance overlaps, and that overlapping variance is not related to crime rates. As we wrote in the paper, it is the non-overlapping variance that is informative, and to see it, we need to include both predictors in the model.

Another way to look at this finding is to collapse the two variables into one variable by subtracting the rates of belief in hell from the rates of belief in heaven, as we did for Figure 2. Creating this single ‘heaven-minus-hell’ variable circumvents any issues of multicollinearity. If we use this new variable as a single predictor for crime rates, the results show the same effect: with no other covariates in the model, standardized beta = .74, 95% Confidence Interval = (.63, .85); with all covariates in the model, standardized beta = .68, 95% C. I. = (.34, .83). This effect is strong and positive: the greater the proportion of people who believe in heaven, relative to the proportion of people who believe in hell, the higher the crime rate. There is no reason to believe the results to be a statistical artifact.

(When looking at the data using this single variable, though, it is important to keep in mind that the twin effects of rates of belief in heaven and hell do depend on each other. Our results do not suggest that, taken on its own, the greater proportion of people who believe in heaven, the greater the crime rate: if all those who believe in heaven also believe in hell, then heaven is not predictive.)

ALTERNATIVE INTERPRETATIONS: This leads nicely into the questions Mr. Loscheider raises about how the findings can be interpreted. Specifically, he suggests that instead of implying that the belief in heaven is tied to higher crime rates, perhaps the finding implies that the lack of belief in Hell is the crucial factor. However, the data do not completely support this explanation—if they did, we would find that the belief in hell, on its own, negatively predicted crime rates. Instead, we only find the negative relationship when the rate of belief in heaven is also added to the regression equation. Most people who don't believe in hell also don't believe in heaven, and what these results suggest is that, for them, their lack of belief is not predictive of anything. Instead, the findings suggest that the lack of belief in hell predicts crime only for those who believe in heaven. If a causal story exists, then a more accurate interpretation may be that for religion to reduce rather than exacerbate unethical behavior, beliefs in supernatural benevolence need to be checked or balanced with beliefs in supernatural malevolence. For those without religion, though, their compliance with ethical behavior is based on different factors.

Discrepancies in the time frames of data collection: Mr. Loscheider also raises the good point that the times differed as to when the data on crime were collected, and when the World Values Surveys assessing the religious belief variables were conducted. He is right; if we had ideal data, we would have all our predictor variables measured at the same time, and crime rate measured at the same time or later.

Nevertheless, the rates of belief in heaven and hell across nations are very stable over time (for those nations (n=19) that had multiple measures, the correlations between the 1981-1997 and 1998-2008 rates were r=0.96 and r=0.96 for heaven and hell, respectively). As a result, we can assume that if all measures were taken at the same time, the pattern of correlations between them would not be substantially different. And there is little to suggest that there would be any systematic difference between the variables that would produce spurious relationships. As a result, the most likely consequence of this decoupling in time of the tested variables is that there will be more noise in the data due to time-specific variance, and therefore lower reliability, and lower correlations between variables, than there would be if the data were contemporaneous. The fact that our findings are robust to that noise suggests that, if anything, the relations we report may be stronger with better data. Should such data become available in the future, it will be worth reanalyzing this relationship. Our expectation is that the current findings would replicate, but the reanalysis should be done.

Though feasibility is still a concern, time differences in the data do raise the possibility of looking to see the relation between our religious variables of interest and crime trends. This would be an avenue for interesting future research, which could shed some light on temporal and perhaps causal patterns.

INTEGRITY OF THE CRIME DATA: Mr. Paul raises questions about the integrity of the crime data used in our analysis. This, we agree, is an important concern. Though the data derive from the United Nations Office on Drugs and Crime (UNODC), which, along with the European Sourcebook, is the gold standard of international crime statistics, such international comparisons are not perfect. In order to construct their international crime databases, the UNODC took pains to maximize the accuracy and comparability of the statistics from different nations, but since these data are collected with the cooperation of local governmental agencies, they are vulnerable to differences in the definition and reporting of crimes. Though the data are widely used, as Paul points out, they have also been criticized for these weaknesses.

The key question is—do these weaknesses compromise the overall effects we report? For several reasons, we don’t believe they do. First, the measure of homicide, that Paul, others and we agree is the most valid, shows the same pattern of results as the overall index averaging the 10 crimes, as well as 7 of the other crimes individually. Second, when we split the sample into local groups of countries (e.g. just European countries or just African countries, and thus mitigate any wide assumed disparities in reporting techniques), then the same pattern of results emerges in each local grouping. Third, re-running the analyses with independently collected homicide data from the European sourcebook yields the same pattern of results (these data correlate strongly with the UN data (r=0.76)—indicating high reliability across measures—but may also suffer from the same weaknesses). Thus, across different crimes, different regions and different data sources, the findings are consistent. Indeed, the consistency may be high enough to outweigh the noise in the data. Fourth, the findings here are completely consonant with lab work looking at the relationship between religious benevolence/malevolence and unethical behavior [1]. This work, though it has its own issues of generalizability, has none of the questionable international comparison concerns of this study. Thus though there is (perhaps considerable) noise in the data, there is methodologically diverse support for the overall conceptual finding.

That all said, homicide rates aside, we do remain open to the possibility that to some degree something other than levels of actual crime—such as reporting biases, or social mores—is also being measured. We welcome alternative, or perhaps additional, explanations as to why these other constructs might co-vary so consistently and in different directions with beliefs in heaven and hell.

RELATIONS WITH RELIGIOSITY: Finally, Mr. Paul falsely ascribes to us the claim “that higher levels of religiosity are associated with higher levels of prosocial behavior.” What we said—that research supports the “claim that religion positively affects normative behavior” —is importantly different. A wealth of experimental research has shown that various different types of religious priming have causal effects on various different measures of prosocial behavior[2-8]. In the review we reference[9], we distinguish the attempts to delineate causal effects using religious priming, from efforts like the ones Paul cites that establish correlations between religiosity and variables of interest. Indeed, these latter studies, which are cited as a challenge to our statement about religion and prosocial behavior, are neither able to establish causation nor are they even about prosocial behavior.

However, the main point that we should reiterate is that our findings are not about comparing religious to non-religious countries, but comparing rates of belief in heaven and hell. For example, as challenge to our findings, Paul indicates that the United States has a high crime rate as well as high rates of belief in God, Heaven and Hell. We would note that obviously one exception—especially a common outlier like the United States—does not invalidate trends found among 67 nations. More importantly, our findings are not about the relationship between crime and absolute levels of religious beliefs—but rather between crime and the divergent effects of two types of religious beliefs. The US has one of the higher disparities between heaven and hell beliefs, and thus it is entirely consistent with the rest of our data that its crime rate is high. Indeed, one of the key aims of the paper is to plumb below the idea of general ‘religiosity’ in order to study more specific beliefs.

We certainly do not take issue with the large amount of data—much of it referenced, and some of it conducted, by Paul—showing relationships between various measures of socioeconomic success and secularization (The work by Barro & McCleary[10] that we cited in the original article showed that the positive relationship between hell beliefs and economic success only exists for developing countries which tend to be the least secularized ones). The evidence for this is overwhelming. However, that is simply not what this study is about. Given the vociferous public debate between atheism and religion, we understand the temptation to read into this paper the familiar ‘secular versus religious’ theme, but we encourage readers to look past it[11].

Thus, despite some disagreements, we have welcomed the conversation about these findings. For the reasons described above, we are confident that the results are not products of statistical artifact or artifice. Given the consistency of the current findings with the body of evidence on the supernatural punishment hypothesis, we have cautiously interpreted the results accordingly. However, as we said in the original article, we remain open to alternative interpretations. We look forward to refining or revising our interpretation in light of future empirical research on the topic.


REFERENCES
1. Shariff AF, Norenzayan A (2011) Mean Gods Make Good People. Int J Psy Rel 21: 85–96.
2. Ahmed AM, Salas O (2011) Implicit influences of Christian religious representations on dictator and prisoner’s dilemma game decisions. J Socio-Econ 40: 242–246.
3. Benjamin DJ, Choi JJ, Fisher G (2010) Religious identity and economic behavior. Unpublished manuscript. Cornell University.
4. Carpenter TP, Marshall MA (2009) An examination of religious priming and intrinsic religious motivation in the moral
hypocrisy paradigm. J Sci Study Relig 48: 386-93.
1. Malhotra D (2010) (When) are religious people nicer? Religious salience and the “Sunday effect” on pro-social behavior. Judgm Decis Mak 5:138-43.
2. Pichon, I., Boccato, G., & Saroglou, V. (2007). Nonconscious influences of religion on prosociality: A priming study. Eur J Soc Psy 37:1032–1045.
3. Randolph-Seng B, Nielsen ME (2007) Honesty: One effect of primed religious representations. Int J Psy Rel 17: 303–315
4. Shariff AF, Norenzayan A (2007) God is watching you: Supernatural agent concepts increase prosocial behavior in an anonymous economic game. Psychol Sci 18: 803-809.
5. Norenzayan A, Shariff AF (2008) The Origin and Evolution of Religious Prosociality. Science 322: 58–62.
6. Barro R, McCleary R (2003) Religion and economic growth across countries. Am Soc Rev 68: 760–781
7. For those who are interested: On its own, a nation’s rate in the belief in God moderately positively predicts homicide rates, but this relationship disappears once nations’ income inequality is taken into account (standardized beta=0.75, p=0.55).