Conceived and designed the experiments: JRH. Performed the experiments: JRH. Analyzed the data: JRH. Wrote the paper: JRH. Critical review of the manuscript and provision of ideas for improvement: BP NS BJM DD. Provision of costing and time data: DD.
The authors have declared that no competing interests exist.
Diagnosis of
We compared cost-effectiveness, as measured by cost per life-years gained and proportion of patients successfully diagnosed and treated, of 33 PCP diagnostic options, involving combinations of specimen collection methods [oral washes, induced and expectorated sputum, and bronchoalveolar lavage (BAL)] and laboratory diagnostic procedures [various staining procedures or polymerase chain reactions (PCR)], or clinical diagnosis with chest x-ray alone. Our analyses were conducted from the perspective of the government payer among ambulatory, HIV-infected patients with symptoms of pneumonia presenting to HIV clinics and hospitals in South Africa. Costing data were obtained from the National Institutes of Communicable Diseases in South Africa. At 50% disease prevalence, diagnostic procedures involving expectorated sputum with any PCR method, or induced sputum with nested or real-time PCR, were all highly cost-effective, successfully treating 77–90% of patients at $26–51 per life-year gained. Procedures using BAL specimens were significantly more expensive without added benefit, successfully treating 68–90% of patients at costs of $189–232 per life-year gained. A relatively cost-effective diagnostic procedure that did not require PCR was Toluidine Blue O staining of induced sputum ($25 per life-year gained, successfully treating 68% of patients). Diagnosis using chest x-rays alone resulted in successful treatment of 77% of patients, though cost-effectiveness was reduced ($109 per life-year gained) compared with several molecular diagnostic options.
For diagnosis of PCP, use of PCR technologies, when combined with less-invasive patient specimens such as expectorated or induced sputum, represent more cost-effective options than any diagnostic procedure using BAL, or chest x-ray alone.
Laboratory-based diagnosis of PCP is a two-step procedure, involving specimen collection and pathogen detection (referred to hereafter as the ‘diagnostic procedure’). Specimens can be collected from oral washes (OW), expectorated (ES) or induced (IS) sputum, tracheal secretions, broncho-alveolar lavage (BAL), or transbronchial biopsies from patients; the latter two require bronchoscopy. Several different methods can be employed for pathogen detection on all specimen types, including immunofluorescence microscopy (IFA),
However, accurate diagnosis of PCP poses multiple challenges. While the procedures to obtain oral washes and sputa are less invasive than that for BAL, they are also less effective at obtaining sufficient numbers of organisms for visualization with diagnostic stains. In contrast, the cost and invasiveness of bronchoscopy and the technical skill it requires render it unfeasible in many areas of the world. Pathological interpretations of stained slides are subjective and nonspecific; sensitivity is dependent on the burden of pathogen in the sample, the specimen type employed, and the skill and experience of the technician examining the sample. Among the pathogen detection methods, the most sensitive is PCR; however, it may be technologically and economically impractical for much of the developing world. Because of these factors, clinicians often use chest x-rays and clinical evaluations as the sole diagnostic method for
This report reviews available diagnostic procedure options for PCP, as well as the cost-effectiveness of each option as a function of procedural cost, sensitivity, and specificity. The outcome measures of interest are the proportion of PCP patients successfully treated and the cost per life-year gained. The analysis is considered from the perspective of the health care payer in developing countries (typically the government). The results should help guide decision-making with respect to diagnostic options for PCP in the developing world.
This analysis is conducted among ambulatory HIV-infected patients in South Africa.
Estimates of sensitivity and specificity of diagnostic procedures used in the model are shown in
Diagnostic | Specimen collection | Sensitivity | Specificity | Cost (USD) |
CXR | None | 0.86 | 0.40 | $40.00 |
DQ | Oral wash | 0.30 | 1.00 | $2.32 |
Expectorated sputum | 0.60 | 1.00 | $2.22 | |
Induced sputum | 0.75 | 1.00 | $8.72 | |
Bronchoalveolar lavage | 0.75 | 1.00 | $77.12 | |
GMS | Oral wash | 0.30 | 1.00 | $4.21 |
Expectorated sputum | 0.52 | 0.95 | $4.11 | |
Induced sputum | 0.70 | 0.96 | $10.61 | |
Bronchoalveolar lavage | 0.82 | 0.98 | $79.01 | |
TBO | Oral wash | 0.30 | 1.00 | $0.93 |
Expectorated sputum | 0.71 | 1.00 | $0.83 | |
Induced sputum | 0.75 | 1.00 | $7.33 | |
Bronchoalveolar lavage | 0.80 | 1.00 | $75.73 | |
CW | Oral wash | 0.30 | 1.00 | $2.94 |
Expectorated sputum | 0.33 | 1.00 | $2.84 | |
Induced sputum | 0.57 | 1.00 | $9.34 | |
Bronchoalveolar lavage | 0.78 | 1.00 | $77.74 | |
IFA | Oral wash | 0.30 | 1.00 | $20.79 |
Expectorated sputum | 0.50 | 1.00 | $20.69 | |
Induced sputum | 0.81 | 1.00 | $27.19 | |
Bronchoalveolar lavage | 1.00 | 1.00 | $95.59 | |
PCR | Oral wash | 0.71 | 0.99 | $8.78 |
Expectorated sputum | 0.85 | 0.99 | $8.68 | |
Induced sputum | 0.94 | 0.99 | $15.18 | |
Bronchoalveolar lavage | 1.00 | 0.94 | $83.58 | |
nPCR | Oral wash | 0.83 | 1.00 | $10.32 |
Expectorated sputum | 0.91 | 1.00 | $10.22 | |
Induced sputum | 1.00 | 1.00 | $16.72 | |
Bronchoalveolar lavage | 1.00 | 0.89 | $85.12 | |
rtPCR | Oral wash | 0.89 | 0.94 | $13.84 |
Expectorated sputum | 0.92 | 0.94 | $13.74 | |
Induced sputum | 0.95 | 0.90 | $20.24 | |
Bronchoalveolar lavage | 0.99 | 0.80 | $88.64 |
CXR: Chest x-ray; DQ: Diff-Quick; GMS: Grocott's Methenamine Silver Stain; TBO: Toluidine Blue O; CW: Calcofluor white stain; IFA: Immunofluorescence; PCR: Polymerase chain reaction; nPCR: nested PCR; rtPCR: real-time (quantitative) PCR.
Estimates of costs include required materials and personnel time (
The value referred to as ‘prevalence’ refers specifically to the prevalence of disease among patients with signs and symptoms of PCP who would normally warrant testing at a given hospital or clinic. It does not refer to the population prevalence of disease. This value will differ regionally; some hospitals or clinics might test all patients with respiratory disease and negative AFB smears, while others will test only patients who have a chest x-ray typical for PCP.
Three models are also presented, with prevalences set at 5%, 20%, and 50%. Treatment failure, whether related to insufficient adherence to treatment or breakthrough infections during treatment to which the patient is adherent, is assumed to occur among 10% of patients (
Treatment costs are based on a single, 21-day regimen with oral CTX (
In studies carried out before the year 2000, median survival time after AIDS diagnosis among patients in developing countries not on antiretroviral therapy was calculated to be approximately one year
An example of model flow with sample values is depicted in
‘Ill patients’ refers to patients with PCP. ‘Well persons’ refers to persons without PCP, regardless of their health status otherwise. Patients successfully treated are assumed to gain one life-year.
All persons diagnosed as PCP-positive (correctly or incorrectly) are assumed to receive a full course of treatment. Treatment failure rates are considered as a combination of failure-to-adhere and breakthrough infection rates (
The proportion of ill patients successfully treated is represented by the number of patients successfully treated divided by the number ill, while the proportion unnecessarily treated is equal to the number of well persons treated divided by the total number of well persons. Total treatment costs are equal to the total number of well persons and ill patients who receive treatment, multiplied by the estimated treatment cost. Finally, the total diagnostic and treatment cost per life-year gained (the cost-effectiveness ratio) is equal to the sum of the total diagnostic costs and the total treatment costs, divided by the number of ill patients successfully treated. The incremental cost-effectiveness ratios of the most effective options were then calculated.
Relapse rates are not considered. Start-up and indirect costs (building costs, laboratory equipment purchase, electricity, training) are also not considered, as they will differ greatly by region and available pre-existing infrastructure.
Sensitivity analyses were performed by varying specific parameters, including treatment costs, treatment failure rates, and costs of diagnostic procedures, over a range of plausible values to determine the impact of uncertainty in the data, and the robustness of results.
Results from the analyses are presented in
Diagnostic | Specimen | % of patients successfully treated | ACER (5% prevalence) | ACER (20% prevalence) | ACER (50% prevalence) |
CXR | None | 77.0% | $1,077 | $270 | $109 |
DQ | Oral wash | 27.0% | $175 | $46 | $20 |
Expectorated sputum | 54.0% | $85 | $24 | $11 | |
Induced sputum | 67.5% | $261 | $68 | $29 | |
Bronchoalveolar lavage | 67.5% | $2,288 | $574 | $232 | |
GMS | Oral wash | 27.0% | $315 | $81 | $34 |
Expectorated sputum | 46.8% | $184 | $48 | $21 | |
Induced sputum | 63.0% | $343 | $88 | $37 | |
Bronchoalveolar lavage | 73.8% | $2,146 | $539 | $217 | |
TBO | Oral wash | 27.0% | $72 | $20 | $10 |
Expectorated sputum | 64.3% | $29 | $10 | $6 | |
Induced sputum | 67.5% | $220 | $57 | $25 | |
Bronchoalveolar lavage | 72.0% | $2,107 | $529 | $213 | |
CW | Oral wash | 27.0% | $221 | $58 | $25 |
Expectorated sputum | 29.7% | $194 | $51 | $22 | |
Induced sputum | 51.3% | $367 | $94 | $39 | |
Bronchoalveolar lavage | 70.2% | $2,218 | $557 | $225 | |
IFA | Oral wash | 27.0% | $1,543 | $388 | $157 |
Expectorated sputum | 45.0% | $923 | $233 | $95 | |
Induced sputum | 72.9% | $749 | $190 | $78 | |
Bronchoalveolar lavage | 90.0% | $2,127 | $534 | $216 | |
PCR | Oral wash | 63.9% | $279 | $72 | $31 |
Expectorated sputum | 76.5% | $231 | $60 | $26 | |
Induced sputum | 84.6% | $363 | $93 | $39 | |
Bronchoalveolar lavage | 90.0% | $1,864 | $468 | $189 | |
nPCR | Oral wash | 74.7% | $279 | $72 | $31 |
Expectorated sputum | 81.9% | $253 | $65 | $28 | |
Induced sputum | 90.0% | $375 | $96 | $40 | |
Bronchoalveolar lavage | 90.0% | $1,901 | $477 | $193 | |
rtPCR | Oral wash | 80.1% | $353 | $90 | $38 |
Expectorated sputum | 82.8% | $339 | $87 | $36 | |
Induced sputum | 85.5% | $483 | $123 | $51 | |
Bronchoalveolar lavage | 89.1% | $2,005 | $503 | $203 |
CXR: Chest x-ray; DQ: Diff-Quick; GMS: Grocott's Methenamine Silver Stain; TBO: Toluidine Blue O; CW: Calcofluor white stain; IFA: Immunofluorescence; PCR: Polymerase chain reaction; nPCR: nested PCR; rtPCR: real-time (quantitative) PCR.
At a disease prevalence of 50%, eight diagnostic procedures had average cost-effectiveness ratios ≤$25 per life-year gained; among these, the most effective (in terms of proportion of PCP patients successfully treated) were IS/TBO, ES/TBO, and ES/DQ (successfully treating 68%, 64%, and 54% of PCP patients, respectively, at $25, $6, and $11 per life-year gained, respectively). Fifteen procedures had an average cost-effectiveness ratio of $26–$100 per life-year gained; among these, the most effective procedures were IS/nPCR, IS/rtPCR, and IS/PCR (resulting in successful treatment of 90%, 86%, and 85% of PCP patients, respectively, at $40, $51, and $39 per life-year gained). Above $100 per life-year gained, the most effective procedures were BAL/nPCR, BAL/IFA, and BAL/PCR, all successfully treating 90% of patients and costing $193, $216, and $189 per life-year gained, respectively. Using a chest x-ray alone for diagnosis resulted in an average cost-effectiveness ratio of $109 per life-year gained and the successful treatment of 77% of patients. Although total costs varied with disease prevalence, relative costs and cost-effectiveness ratios did not (
The scatterplot in
Triangles represent procedures involving oral washes; circles represent procedures involving expectorated sputum; lines represent procedures involving induced sputum; and squares represent procedures involving BAL. All white data points outlined in black indicate procedures using the Diff-Quick test (1–4); red indicates procedures using GMS (5–8); orange indicates procedures using TBO (9–12); light green indicates procedures using calcofluor white (13–16); dark green indicates procedures using IFA (17–20); black indicates procedures using PCR (21–24); purple indicates procedures using nPCR (25–28); and blue indicates procedures using rtPCR (29–32).
We evaluated incremental cost-effectiveness among diagnostic options which resulted in successful treatment of at least 2/3 (67%) of PCP patients. After excluding both strongly dominated (less effective and more expensive) and weakly dominated (equally effective but more expensive, or equal in cost but less effective) options, five procedures remained for inclusion in an incremental cost-effectiveness analysis: induced sputum with TBO, PCR, or nPCR; and expectorated sputum with nPCR or rtPCR (
Specimen collection and diagnostic procedure | % of patients successfully treated | Average cost per life-year gained | Cost per additional life-year gained, compared with next-least-effective procedure |
IS/TBO | 0.68 | $25 | $N/A |
ES/nPCR | 0.82 | $28 | $43 |
ES/rtPCR | 0.83 | $36 | $804 |
IS/PCR | 0.85 | $39 | $156 |
IS/nPCR | 0.90 | $40 | $60 |
IS: Induced sputum; ES: expectorated sputum; TBO: Toluidine Blue O; PCR: polymerase chain reaction; nPCR: nested PCR; rtPCR: real-time (quantitative) PCR.
Variations in the cost of the diagnostic procedure had the most impact on cost per life-year gained in sensitivity analyses. Reducing the cost of the diagnostic procedures by 50% led to an approximate 50% reduction in cost per life-year gained, while doubling it led to approximately a two-fold increase in cost per life-year gained (
Three metrics are relevant in this analysis for decision-making and policy concerning diagnostic testing for PCP: (a) proportion of PCP patients successfully treated, (b) proportion of well persons unnecessarily treated, and (c) the total diagnostic and treatment cost per life-year gained. An ideal test will maximize the first metric and minimize the second, at the smallest – and most feasible, for the implementing clinic or geographic region under consideration – value of the third. Because all laboratory-based diagnostic procedures considered in this analysis were highly specific, the effect of (b) is negligible for this analysis; thus, we presented the results as a function of (a) and (c).
Our results indicate that PCR methodologies are so sensitive that, specimen type notwithstanding, they represent the most cost-effective diagnostic options for PCP. When PCR methodologies are available, they mitigate the need for obtaining highly invasive specimens, such as BAL, which increase procedural sensitivity at substantial increases in cost. However, if both PCR and machinery for sputum induction are unavailable at a given site, the next-best option could be ES/TBO, which is relatively inexpensive and simple in terms of specimen collection and laboratory requirements for diagnosis. Although the use of chest x-ray alone for diagnosis can lead to the successful detection and treatment of high proportions of patients, the cost per life-year gained exceeds that of other equally-sensitive or more sensitive methods for diagnosing disease.
In general, the decision about which test is most useful in a given region will depend on the estimated prevalence of PCP among persons tested, local technical capacity, and available financial resources. Individual patient characteristics may affect decision-making, too; in an already-intubated patient, a BAL will be meaningfully cheaper than it would be among non-intubated patients, making the increased sensitivity in specimen collection more economical as well as practical (as an intubated patient or an infant will be unable to produce sputum). In addition, BAL might facilitate the detection of other respiratory pathogens besides
There are several limitations to this analysis. First, data were not available on the sensitivity and specificity of all diagnostic procedures, creating a need to estimate some values. Even for procedures for which data were available, the degree of experience of an administering clinician or laboratory technician could affect the test's sensitivity or specificity. Second, indirect costs are not included in the model. The buildings, equipment, and technical know-how needed to carry out more advanced molecular diagnostics such as PCR are not currently in place in all countries. Where this capacity does exist, it may be unevenly distributed geographically and might not be accompanied by appropriate quality assurance measures. Start-up costs to implement these technologies could be prohibitive for some low-income countries, and in these areas comparisons of the cost-effectiveness of the various staining methods might be more useful than considerations about which PCR methodology is optimal. Finally, we did not account for differing diagnostic or treatment costs in different countries or among different patient groups, which could affect overall cost or cost-effectiveness of different diagnostic options. However, it is worth noting that, although the costs of all procedures might differ by country, the relative cost of procedures is unlikely to differ greatly.
Existing international guidelines call for CTX prophylaxis of PCP in patients whose CD4+ T cell counts drop below 350 cells/mm
The cost-effectiveness of diagnostic testing improves in areas of higher disease prevalence; testing might become prohibitively expensive in areas with very low prevalence of disease. However, a diagnostic protocol that might seem financially unfeasible for certain regions might be more feasible than suspected if the prevalence of disease can be increased among the patients selected for testing. One way to optimize test utility is to use a clinical algorithm that improves the pre-test probability without incurring substantial numbers of false negatives. Although no such algorithm has been formally defined for PCP, clinical differences do exist between HIV-infected patients with PCP compared with other pneumonias; PCP patients have a more subacute onset of disease, ground glass infiltrates on CXR
For any disease, when the cost of diagnosis exceeds the cost of treatment (such as with PCP), the cost-effectiveness of empiric diagnosis and treatment is directly proportional to the gap between the diagnostic and treatment costs; thus, when treatment costs are very low, it's nearly always more cost-effective to diagnose and treat patients empirically. In addition, because international guidelines call for at-risk patients to be on ART and CTX prophylaxis, the occurrence of PCP in a patient likely represents a failure of the local health system to provide sufficient opportunities for HIV patient care and treatment, an inability by the treating clinic to meet these standards, an inability by the patient to adhere to the recommended treatment regimen, or drug failure. Thus, one could argue that efforts should be focused on improving access to care for HIV patients or adherence to the standards laid out in international guidelines with respect to ART and CTX treatment, rather than on diagnosing the precise etiology of infections that could otherwise have been prevented. This is a valid argument and such efforts should be supported. However, given the suboptimal conditions that currently exist with respect to meeting these guidelines, there are benefits to accurate diagnosis, including improvements in the understanding of the true prevalence of disease, which is worthwhile for the purposes of prevention, control, and allocation of resources. This analysis is not intended to discourage PCP prophylaxis or diagnosis and treatment among symptomatic patients in the absence of a laboratory-based diagnosis, but rather to provide a basis for decisions on diagnostics for PCP, should an institution desire to implement diagnostic procedures. For these institutions, particularly in situations of high disease prevalence, we demonstrate that the elevated sensitivity and specificity of diagnosis enabled by the use of PCR technologies could justify the additional costs of obtaining and using them. A rough calculation demonstrates the power of replacing microscope-based technologies with PCR technologies for the diagnosis of PCP: in South Africa, the adult HIV infection rate is reported at 20%, with an estimated half a million new infections
Diagnostic procedural decisions cannot, in practice, be simplified to numbers alone. Assuming clinicians were aware of the diagnostic qualities of each test, they could make decisions outside of the framework presented here, such as conducting sequential tests (for example, a highly sensitive test followed by a highly specific test) for diagnostic purposes. In addition, we realize that most clinicians do not have an array of diagnostic options at hand, and if a diagnostic protocol is to be implemented, it will be done at a clinic, hospital, or regional level. However, examples of molecular diagnostic technologies in resource-limited settings are increasingly reported, for example with tuberculosis diagnosis
Model inputs: Estimated personnel and time requirements and associated costs for specimen collection options for
(DOC)
Model inputs: Personnel and time requirements and associated costs for laboratory procedures for diagnosis of
(DOC)
Model inputs: cost of treatment, treatment failure rate, and prevalence of disease in the population.
(DOC)
Sensitivity analysis: cost per life-year gained with variations in diagnostic procedure cost, sensitivity, specificity, treatment failure rates, and treatment costs. *Neither sensitivity nor specificity was increased beyond a value of 1.00. Procedures which in the base model were 0.90 or greater were capped at 1.00. CXR: Chest x-ray; DQ: Diff-Quick; GMS: Grocott's Methenamine Silver Stain; TBO: Toluidine Blue O; CW: Calcofluor white stain; IFA: Immunofluorescence microscopy assay; PCR: Polymerase chain reaction; nPCR: nested PCR; rtPCR: real-time (quantitative) PCR; Expect. sputum, expectorated sputum.
(DOC)
Estimated salaries for laboratory workers, health care workers, and clinicians involved in patient care and diagnosis of PCP.
(DOC)
The authors would like to thank Drs. Mark Lindsley, S. Arunmozhi Balajee, and Graeme Meintjes for their thoughtful input and manuscript review. These data were presented in part previously at the Infectious Diseases Society of America conference in October 2010 (Vancouver), and published as abstract in the proceedings, as poster #1048.
The findings and conclusions in this report are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.