Reader Comments

Post a new comment on this article

GIGO

Posted by vanhowe on 16 Feb 2010 at 17:12 GMT

Dear Editors:

Cost-utility and cost-effectiveness analyses are tools that allow public health policy makers to determine the best use of resources for a particular goal. Any analysis, however, is only as good as its assumptions, so the acronym GIGO (garbage in, garbage out) often applies. Sometimes analyses are developed with a particular result in mind and assumptions are crafted to generate the desired result. This is can also be GIGO (garbage in, gospel out). Evidence suggests that the cost-utility analysis by Samson et al.[1] may fall into the latter category.

The cost-utility model presented by Sansom and colleagues is based on six basic assumptions, all of which are suspect.

First, it was assumed that the lifetime risk for a male in the United States to become HIV-infected was 1.87% (1 in 53). By race and ethnicity, the risk was assumed to be 0.96% (1 in 104) in whites, 2.88% (1 in 34) in Hispanics, and 6.23% (1 in 16) in blacks. In a national probability sample studied by the University of Chicago of men 18 to 59 years of age in the mid 1990s, 9 of 2577 (0.35%, 1 in 286) men were HIV-infected.[2] While some of these men were young and still had time to become infected, it is inconsistent with the five-fold increase in prevalence used by the authors. The prevalence has not increased five-fold since the mid 1990s. Consequently, this assumption is dubious.

Second, it was assumed that the efficacy of circumcision for the prevention of heterosexually transmitted HIV was 60% over a lifetime. This larger assumption is based on several other secondary assumptions.

The first of these secondary assumptions is that 60% efficacy, which the highest efficacy of the three randomized clinical trials (RCTs) undertaken in Africa,[3-5] is valid. There are several reasons to believe that it is not. First, lead-time bias was built in to the study design of all three studies. This bias would overestimate the treatment effect. Likewise, built-in expectation bias and selection bias would overestimate the treatment effect. Reporting the results using a modified intention-to-treat methodology would overestimate the treatment effect. The studies also had a high attrition rate compared to the number who developed to outcome of interest. Finally, all three studies were halted early. This would amplify the lead-time bias. Also, studies that are halted early, especially when the number of events is small (as was the case in these RCTs), often show implausibly large treatment effects.[6] The authors twice noted that the results of RCTs were “remarkably consistent.” No argument on that point: the number of circumcised men who because HIV-infected in the three RCTs were 20, 22, and 22, while the number of normal men who became HIV-infected was 49, 47, and 45. The question is whether this consistency is beyond remarkable. Consequently, this assumption is dubious.

The second of these secondary assumptions is that 60% reduction in risk would be seen over a lifetime. The RCTs were originally were designed to follow men for 24 months, but the studies were interrupted before completion. It is foolhardy to assume that a trajectory based on less than two years of data will continue over 40 years. It is unknown whether the pattern continues in a linear fashion, an exponential fashion, or if circumcision merely delays infection and the overall prevalence is the same over a lifetime. While the RCTs performed in South Africa claimed a 60% reduction in HIV infections, in the South Africa’s general population circumcised and normal men have similar HIV-infection rates.[7,8] This would indicate that the 60% reduction in HIV-infection risk is a short-term phenomenon. The lack of impact on the general population is best explained by risk compensation.[9,10] Believing that circumcision provides additional protection, the circumcised male will use condoms less frequently and engage in more risky behaviors. This may be exacerbated by that fact that in Africa the money previously earmarked for condoms has been diverted to circumcision, making fewer condoms available. In this scenario, programs to implement circumcision may increase the overall HIV-infection rates. The assumption that any reduction in risk lasts a lifetime has no factual basis.

The third of these secondary assumptions is that the results seen in Africa in high-risk, well-compensated men who were willing to be circumcised would apply to a program of circumcising all males in the United States regardless of their risk of HIV-infection. To support this assumption the authors cite one study of men whose sexual partner was HIV-positive,[11] and another study from a sexually transmitted disease clinic.[12] This would be reasonable if the authors were recommending circumcision for those at the greatest risk for HIV-infection, but the authors are suggesting that circumcision would have 60% efficacy across all risk strata. There is no evidence to support that assumption. When the observational studies of the association between circumcision status and HIV-infection are separated into studies of high-risk populations and general populations, the association, if one can be found, in the studies of general populations is much lower.[13] It also appears as if the authors have cherry-picked the studies performed in the United States to support their assumption. The largest study of HIV-infection risk among heterosexual men in the United States was performed in San Francisco. They found no association between circumcision status and HIV infection (OR=0.93, 95%CI=0.83-1.05).[14] So the assumption that these results can be applied to a general population or to the United States is not credible.

The fourth of these secondary assumptions is that the results in adult circumcision are applicable to infant circumcision. To date, there have been no observational studies to indicate that infant circumcision has any association with reduced heterosexually-transmitted HIV-infection rates. To the contrary, the United States, which has a high rate of infant circumcision, has an incidence of heterosexually-transmitted HIV-infection several times greater than that seen in Europe and Japan, where circumcision is rare.[15] There is no evidence to support the assumption that the results of adult circumcision trials in Africa apply to infants.

Third, it was assumed that blacks have a lower circumcision rate than whites. There is evidence that contradicts this. One study found equal rates,[16] while two studies have found higher rates in blacks.[14,17] The fact that blacks have higher circumcision rates than whites, yet have a risk of heterosexually-transmitted HIV-infection that is nearly five times higher, suggests that circumcision is not an important factor in HIV-prevention. Rather than being 60% effective, this indicates that circumcision has failed to protect black males from HIV infection. Finding that circumcision would have the greatest impact on blacks, already the most circumcised sector of the population, is counterintuitive. Before contemplating a model of infant circumcision, the lack of correlation between the circumcision rate and heterosexually-transmitted HIV-infection rate between ethnic groups needs to be explained. Once this explanation has been identified, resources should be directed to the factors that put blacks at greater risk. The lack of any correlation between HIV-infection rates and circumcision rates in the ethnic groups studied also undermines the assumption that circumcision would have 60% efficacy. If circumcision were effective, the numbers should have reflected it by now.

Fourth, it was assumed that a 3% discount was appropriate. Most cost-utility analyses with either use a 5% discount rate or report their results using both. Discount rates are used to measure opportunity costs. The money spent on infant circumcision is no longer available for other purposes. If the money were available, there would be the opportunity to invest it and earn additional money. In the model proposed by the authors, the average time of investment is over 30 years (the age at which HIV-infections is typically diagnosed). Even in a recession, a 3% return on a long-term investment would not be an attractive option. Consequently, a 3% discount rate is not realistic. A conservative long-term investment should yield at least 5%, so a 5% discount rate would be the most reasonable of conservative assumptions. One could easily argue that discount rates of 7% to 10% in a 30-year investment are more appropriate. A lower discount rate, as opposed to a higher discount rate, favors earlier interventions aimed at delayed benefits.

Fifth, it was assumed that infant circumcision had no impact on the length of stay of the perinatal hospitalization. It has been demonstrated that one in six males circumcised as newborns will spend an extra day in the hospital.[16]

Sixth, it was assumed that infant circumcision had no complications. Previous cost-utility analyses have found that mild, more common events have the most impact on the cost-utility of infant circumcision.[18,19] For example, 5% of circumcised boys develop meatal stenosis that requires a meatotomy.[20] At $1500 per procedure, this adds $64.41 (3% discount for 5 years) to the cost of each circumcision. MRSA infections are 12 times more common in infants that are circumcised and outbreaks increasing in frequency.[21,22] Heterosexually-transmitted HIV infection is a relatively rare event, but it may of similar frequency as botched circumcisions in which too much skin is removed, part of the glans is amputated, or there is a serious infection or bleeding. The cost of treating these complications, the multi-million dollar malpractice settlements, and the impact on health need to be considered. Death is a well-documented complication of circumcision that takes all of the life-years (quality adjusted and otherwise) away from an infant.[23] If rare catastrophic events are to be considered on one side of the ledger, they also need to be considered on the other.

Dubious assumptions aside, Sansom and colleagues fail to address several pertinent issues related to their analysis. First, they do not attempt to put their analysis into context of current prevention strategies. To make the case that circumcision is a reasonable option, circumcision needs to be compared to limiting the number of sexual partners, using condoms, early treatment of sexually transmitted infections, and secondary prevention measures such as treating HIV-positive individuals with anti-retroviral therapy (ART). All these options are known to be more effective, less invasive, and less expensive than circumcising all infants.

An analysis of how circumcision would compare to the recent vaccine trial results would be of interest. A vaccine would help men having sex with men (MSM), for whom circumcision has shown no association with HIV-infection risk,[24] and women, who have been shown to be at 50% greater risk for HIV-infection if their male partner is HIV-infected and circumcised.[25]

Second, the impact of ART, which is included in the cost of treatment, may not be properly accounted for. The second major assumption is based on studies performed in an environment where HIV surveillance and ART are not well utilized. An individual on ART who is HIV-infected but has no active sexually transmitted infection is considered to be at no risk of transmitting the virus through sexual contact.[26] Consequently, the treatment is a more effective preventive than the intervention the authors propose.
Aggressive HIV surveillance and ART for those who are HIV-infected would have the advantage of targeting those at highest risk rather than specific racial groups. The impact of this secondary prevention would also be immediate.

Third, high-risk behavior does not occur in infancy. The authors need to demonstrate that this intervention is important enough to violate the child’s right to bodily integrity and that there is a problem with waiting until the child is old enough to consent to the procedure. With more effective, less expensive, less invasive options available, the individual should be allowed to make this choice.

Fourth, the rates of heterosexually transmitted HIV are greater in women. Male circumcision may only exacerbate this.[25] One study has found that female circumcision is associated with a lower risk of HIV infection.[27] Based on this study, the authors should be encouraged to perform a similar analysis to assess the cost-effectiveness of infant female circumcision in preventing HIV infection.

Using a model similar to the one constructed by the authors, one could make the argument that neonatal penectomy would have 100% efficacy against sexually transmitted HIV, other sexually transmitted infections, and unwanted pregnancies. If the complications and loss of function and sexual enjoyment are left out of the analysis, as the authors have done with their circumcision model, one could argue that penectomy is a better option than circumcision for preventing HIV infection. It is not clear why the authors limited their analysis to amputation of the foreskin.

To construct a cost-utility model that is most favorable to infant circumcision, the model would assume the highest efficacy rate, the highest prevalence of HIV, no complications from circumcision, and the lowest discount rate. The authors of this cost-utility did all of these things. Too bad none of their assumptions are valid.


References

1. Sansom SL, Prabhu VS, Hutchinson AB, An Q, Hall I, Shrestha RK, Lasry A, Taylor AW. Cost-effectiveness of newborn circumcision in reducing lifetime HIV risk among U.S. males. PLoS One 2010; 5(1): e8723.

2. Laumann EO, Masi CM, Zuckerman EW. Circumcision in the United States: prevalence, prophylactic effects, and sexual practice. JAMA 1997; 277: 1052-7.

3. Auvert B, Taljaard D, Lagarde E, Sobngwi-Tambekou J, Sitta R, Puren A. Randomized, controlled intervention trial of male circumcision for reduction of HIV infection risk: The ANRS 1265 Trial. PLoS Med 2005; 2(11): e298

4. Bailey RC, Moses S, Parker CB, Agot K, Krieger JN, Williams CFM, Campbell RT, Ndinya-Achola JO. Male circumcision for HIV prevention in young men in Kisumu, Kenya: a randomised controlled trial. Lancet 2007; 369: 643-56.

5. Gray RH, Kigozi G, Serwadda D, Makumbi F, Watya S, Nalugoda F, Kiwanuka N, Moulton LH, Chaudhary MA, Chen MZ, Sewankambo NK, Wabwire-Mangen F, Bacon MC, Williams CFM, Opendi P, Reynolds SJ, Laeyendecker O, Quinn TC, Wawer MJ. Male circumcision for HIV prevention in men in Rakai, Uganda: a randomised trial. Lancet 2007; 369: 657-66.

6. Montori VM, Devereaux PJ, Adhikari NK, Burns KE, Eggert CH, Briel M, Lacchetti C, Leung TW, Darling E, Bryant DM, Bucher HC, Schünemann HJ, Meade MO, Cook DJ, Erwin PJ, Sood A, Sood R, Lo B, Thompson CA, Zhou Q, Mills E, Guyatt GH. Randomized trial stopped early for benefit: a systematic review. JAMA 2005; 294: 2203-9.

7. Connolly C, Shanmugam R, Simbayi LC, Nqeketo A. Male circumcision and its relationship to HIV infection in South Africa: Results of a national survey in 2002. S Afr Med J 2008; 98: 789-94.

8. Garenne M. Long-term population effect of male circumcision in generalised HIVepidemics in sub-Saharan Africa. Afr J AIDS Res 2008; 7: 1-8.

9. Blower SM, McLean AR. Prophylactic vaccines, risk behaviour change, and the probability of eradicating HIV in San Francisco. Science 1994; 265: 1451-4.

10. Richens J, Imrie J, Copas A. Condoms and seat belts: the parallels and the lessons. Lancet 2000; 355: 400-3.

11. Warner L, Ghanem KG, Newman DR, Macaluso M, Sullivan PS, Erbelding EJ. Male circumcision and risk of HIV infection among heterosexual African American men attending Baltimore sexually transmitted disease clinics. J Infect Dis 2009; 199: 59-65.

12. Telzak EE, Chiasson MA, Bevier PJ, Stoneburner RL, Castro KG, Jaffe HW. HIV-1 seroconversion in patients with and without genital ulcer disease. Ann Intern Med 1993; 119: 1181-6.

13. Van Howe RS. Circumcision and HIV infection: review of the literature and meta-analysis. Int J STD AIDS 1999; 10: 8-16.

14. Mor Z, Kent CK, Kohn RP, Klausner JD. Declining rates in male circumcision amidst increasing evidence of its public health benefit. PLoS ONE 2007; 2(9): e861.

15. Storms MR. AAFP fact sheet on neonatal circumcision: a need for updating. Am Fam Physician 1996; 54: 1216, 1218.

16. Mansfield CJ, Hueston WJ, Rudy M. Neonatal circumcision: associated factors and length of hospital stay. J Fam Pract 1995; 41: 370-6.

17. O'Brien TR, Calle EE, Poole WK. Incidence of neonatal circumcision in Atlanta, 1985-1986. South Med J 1995; 88: 411-5.

18. Ganiats TG, Humphrey JB, Taras HL, Kaplan RM. Routine neonatal circumcision: a cost-utility analysis. Med Decis Making 1991; 11: 282-93.

19. Van Howe RS. A cost-utility analysis of neonatal circumcision. Med Decis Making 2004; 24: 584-601.

20. Van Howe RS. Incidence of meatal stenosis following neonatal circumcision in a primary care setting. Clin Pediatr (Phila) 2006; 45; 49-54.

21. Nguyen DM, Bancroft E, Mascola L, Guevara R, Yasuda L. Risk factors for neonatal methicillin-resistant Staphylococcus aureus infection in a well-infant nursery. Infect Control Hosp Epidemiol 2007; 28: 406-11.

22. Van Howe RS, Robson WLM, The possible role of circumcision in newborn outbreaks of community-aassociated methicillin-resistant Staphylococcus aureus. Clin Pediatr (Phila) 2007; 46: 356-9.

23. Paediatric Death Review Committee: Office of the Chief Coroner of Ontario. Circumcision: a minor procedure? Paediatr Child Health 2007; 12: 311-2.

24. Millett GA, Flores SA, Marks G, Reed JB, Herbst JH. Circumcision status and risk of HIV and sexually transmitted infections among men who have sex with men. JAMA 2008; 300: 1674-1684.

25. Wawer MJ, Makumbi K, Kigozi G, Serwadda D, Watya S, Nalugoda F, Buwembo D, Ssempijja V, Kiwanuka N, Moulton LH, Sewankambo NK, Reynolds SJ, Quinn TC, Opendi P, Iga B, Ridzon R, Laeyendecker O, Gray RH. Circumcision in HIV-infected men and its effect on HIV transmission to female partners in Rakai, Uganda: a randomised controlled trial. Lancet 2009; 374: 229-37.

26. Vernazza P. La prévention du sida devient plus simpl, mais aussi plus complexe! Bull Med Suisses 2008; 89: 163-4.

27. Stallings RY, Karugendo E. Female circumcision and HIV infection in Tanzania: for better or for worse?[abstract] Third International AIDS Society Conference on HIV Pathogenesis and Treatment. Rio de Janeiro, July 25-27, 2005.

No competing interests declared.