Reader Comments

Post a new comment on this article

The importance of study design vs. analysis technique: A response to Talbott

Posted by rugorugo on 29 Jul 2007 at 21:00 GMT

Dear Editors,

Much criticism has been put forth to Talbott paper published in PLoS ONE on June 20th [1]. I would also like to add to the list by commenting on the importance of study design against the analysis technique.

Randomised Controlled Trials (RCTs) rank high in the hierarchy for evidence in epidemiological studies and thus provide strong evidence for association. The Orange farm trial in South Africa [2], Kisumu trial in Kenya [3] and Rakai trial in Uganda [4] have shown that male circumcision prevents HIV acquisition by 60%. Given the fact that men have a tendency towards multiple sexual partners (CSW inclusive) they can be considered as a link of HIV transmission to the general population. Thenceforth, empirically, it can be argued that, circumcision is the key to slowing down the HIV epidemic.

Cross-country studies are ecological studies, ranking low in the hierarchy for epidemiological evidence due to unobserved confounding (among other factors) which cannot be controlled whatever robust the statistical method is. Due to randomization, unobserved confounding is not a problem in RCTs. Hence, strong evidence cannot be drawn from ecological studies.

Regression methods are based on assumptions and cannot explain relationships to a tune of 100%. Looking at the two multiple regression models that contain CSW and Muslims (Table 4) in the Talbott study, the R squared is 48-49% [1] implying that the two models do not explain over 50% of the association and thus they are not the best models.

It is not uncommon to have a problem of missing data in secondary data analysis like in the study by Talbott. Usually this, results into selection bias affecting the generalizability of the results [5]

Earlier studies showed that HIV prevalence is low in Muslim countries [6], but these were ecological studies with their inborn problems of fallacy. This means that, on individual basis, being a Muslim is not equal to being protected from HIV. Because circumcision is an important ritual in Muslims, these studies however, set forth the hypothesis of male circumcision against HIV transmission that was proven by the three RCTs. Nevertheless, the variable Muslim might not be a good proxy for circumcision despite the good correlation reported before. Other factors such as ethnicity, number of males, age, location (which have not been considered before) may cause the insignificant correlation if controlled in multiple regression. Ideally, Talbott should have correlated the percentage of circumcised people in the country (which is difficult to get anyway) to HIV prevalence.


Therefore the conclusion, “ This paper provides strong evidence that when conducted properly, cross country regression data does not support the theory that male circumcision is the key to slowing the AIDS epidemic" [1] is not warranted. There is weak evidence, which needs to be revisited. Besides, in the study the number of Muslims appear to show a negative correlation to HIV prevalence after controlling for number of infected CSW but fails to reach standards of significance which is common with small sample size. Talbott did not provide the sample size for Muslims in his study and judging from the 77 countries (with Iran and Morocco only, leaving out other highly populated Islamic states) it is relatively small.

In conclusion, the Muslim-HIV epidemic association in the paper is over interpreted and thus the paper is flawed in the aspect of circumcision and HIV epidemic but it has a merit on eliciting the number of infected CSW as an important determinant for HIV epidemic. Unless the study is conducted using a strong methodology and a good number of Bradford-Hill criteria are met, it is better to avoid words like “strong evidence…” One should be cautious, otherwise it is not necessary to draw conclusions from weaker study designs if the evidence is already established by stronger study designs. Let us not be “hypnotized” by complex statistical techniques but appreciate and interpret results in real world biological systems.

Rugola Mtandu
University of The Witwatersrand, School of Public Health

References

1. Talbott JR (2007) Size Matters: The Number of Prostitutes and the Global HIV/AIDS Pandemic. PLoS ONE 2(6):e543

2. Auvert B, Taljaard D, Lagarde E, Sobngwi-Tambekou J, Sitta R, et al. (2005) Randomized,controlled intervention trial of male circumcision for reduction of HIVinfection risk: The ANRS 1265 trial.PLoS Med 2(11): e298.

3. Bailey RC, Moses S, Parker CB, Agot K, Maclean I, et al. (2007). Male circumcision for HIV prevention in young men in Kisumu, Kenya: a randomised controlled trial. Lancet 369 (9562): 643–56


4. Gray RH, Kigozi G, Serwadda D, Makumbi F, Watya S, et al. (2007). Male circumcision for HIV prevention in men in Rakai,Uganda: a randomised trial. Lancet 369 (9562): 657–66

5. Schoenbach VJ, Rosamond WD, (2000) Understanding the Fundamentals of Epidemiology an evolving text. 2nd ed. Chapel Hill, North Carolina, pp.306, 540-544.

6. Gray, PB (2004) HIV and Islam: is HIV prevalence lower among Muslims? Social Science & Medicine 58(9): 1751-1756