Introduction

Canada is experiencing a major shortfall in deceased organ donations similar to many other countries [1]. In 2011, 3361 Canadians were waiting for a solid organ transplant while 304 patients died waiting [2]. Several initiatives are underway in many countries to increase organ donation including registration of intent to donate, presumed consent policies, financial incentives for donors, and relaxed criteria for donation [3], [4].

Health administrative data contain information collected passively from government and healthcare providers for managing patients [5]. However, they were not designed for research purposes and prone to data entry errors [6]. In addition, there may also be no financial incentive for physicians to provide accurate diagnostic data for billing information [7]. Previous database research has shown that some diagnostic codes and physician claims can be used reliably to identify patients with a certain condition [7], [8], but not others [9]–[11].

Many organ procurement organizations keep detailed records of their activities that require laborious manual chart reviews at multiple hospitals [12]. Obtaining this information from large healthcare administrative databases may be a cost-effective approach and may even provide additional data elements. Further, administrative databases can often be linked with other datasets to provide additional data elements. We conducted a detailed PubMed search with keywords “organ donor”, “organ donation” and “validation” retrieved no prior studies in any jurisdiction describing the validity of algorithms to identify deceased organ donors within administrative healthcare databases.

With respect to data requirements in this field, what is needed is accurate data on: 1) all deaths (data which does exist in many jurisdictions), 2) all deaths eligible for deceased organ donation, 3) all eligible deaths where the family was approached for deceased organ donation, 4) those who died where organs were procured for transplantation (and which organs), and 5) those who died and had organ procured and successfully transplanted into a recipient (and which organs/number of organs per donor). In this study we sought to determine whether administrative databases can be a reliable source to identify step 5) individuals who became a deceased organ donor and those organs transplanted into a recipient.

To identify such donors and the organs transplanted we tested different physician billing algorithms in the provincial administrative databases of Ontario, Canada. We hypothesized that physician billing claims are a reliable source of data to identify deceased donors because organ procurement is a clearly defined procedure. Thus, the presence of a physician billing claim associated with organ procurement should serve as a valid marker for whether the person was an organ donor. The ultimate goal was to determine whether these algorithms can be reliably used in future policy-making, health services delivery planning, quality improvement and research.

Methods

Results

The total number of deaths reported in the Registered Persons Database during the study period (in-hospital and out-of-hospital) was 458,074. During this period, there were also 988 unique deceased organ donors identified using the reference standard, including 848 unique deceased kidney donors, 688 liver donors, 298 lung donors, 234 heart donors, 150 pancreas donors, and 6 small intestine donors. Of these donors, 15 (1.8%) did not have a death date and 6 (0.6%) did not have an associated physician billing code and therefore could not be identified in the administrative data. All 988 unique deceased organ donors were counted in the analysis.

The billing code for transplant counseling performed poorly (sensitivity of 7% and positive predictive value of 11%) and was excluded from further consideration. There were 10 billing codes that successfully identified at least one deceased organ donor (See Table S2). These codes were combined into an algorithm, which identified an organ donor if any one of the 10 codes were present. Overall, this algorithm identified 745 individuals as deceased organ donors. Algorithm sensitivity was 75.4% (95% confidence interval (CI): 72.6%, 78.0%) and the positive predictive value was 77.4% (95% CI: 74.7%, 80.0%) (Table 1). As expected, specificity and negative predictive value were near 100% given the low prevalence of organ donation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Validity measures using chart abstraction as the reference standard for the identification of deceased organ donor cases by the best-performing billing coding algorithms.

https://doi.org/10.1371/journal.pone.0070825.t001

The codes operated poorly for the detection of organ-specific donation. Algorithms for the identification of heart donors had the best sensitivity (66.7%), followed by lung (55.7%), liver (35.6%), and kidney (23.6%) donors. Algorithms for the identification of kidney donors had the best positive predictive value (71%), followed by liver (66%), lung (52%), and heart (37%) donor algorithms. Physician billing codes were not able to identify any pancreatic and small intestine donors.

Additional Analyses

Attempts to improve our algorithms by restricting the cohort to deaths that were more likely to be eligible for donation were ineffective. First, we were not able to identify 15% of our reference standard donors when we restricted our cohort to hospitalized deaths. Second, we were not able to improve the positive predictive value when we restricted our cohort to all deaths (in and out-of-hospital) and at least one code relevant to mechanical ventilation at the time of death (results not shown).

According to CORR, there were 3126 deceased donors (with at least one organ transplanted) in Ontario during the period of 1991 to 2010 (Figure 1). With our best-performing algorithm (consisting of any of 10 billing codes), we identified 2872 deceased donors during the same period. Our algorithm identified a significantly higher number of heart transplants and much lower rates of kidney transplants and liver transplants (See Figure S1). The number of lung donors did not appear to be significantly different. During the pre-validation period (1991–2005), the billing algorithm identified 474 lung donors compared to an expected 529 donors.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Number of deceased organ donors per year from 1991 to 2010 in Ontario.

Expected (blue line) represents the number of deceased organ donors as reported by the Canadian Organ Replacement Registry. Identified (red line) represents the number of deceased organ donors as reported by our physician billing algorithm. Reference (green line) represents the number of deceased organ donors as reported by manual chart reviews from 2006 to 2010.

https://doi.org/10.1371/journal.pone.0070825.g001

Discussion

We analyzed all deaths that were registered in administrative health databases in Ontario and identified organ donation with an algorithm using the physician billing databases. We recommend that primary data abstraction preferentially be used to identify deceased organ donors, particularly for the detection of organ-specific donors. When such primary data are not available, the limitations of physician billing claims should be considered whenever they are used. We found that the estimated number of decedents labeled as organ donors each year by our algorithm was similar to that obtained from the national registry. Our algorithm identified deceased organ donors in our province with 75.4% sensitivity and 77.6% positive predictive value compared to the reference standard of actual donations that were recorded in the procurement agency's database and confirmed through manual chart review. This level of sensitivity is similar for algorithms finding other conditions such as deaths (60% sensitivity; 94.8% positive predictive value) in Japanese claims data [17]. When used, methods to correct for misclassification can be considered [18].

The algorithms were largely ineffective at identifying organ-specific donors, highlighting the need for better coding of organ donation in administrative data. For example, the codes were not able to capture any pancreatic and small intestine donors. Conversely, we identified a larger number of deceased heart donors using administrative data compared to the national registry. It is important to note that deceased heart and lung donors share a common code, “M157” (donor heart and lung removal) that could have resulted in a higher number of false positives in our analysis. Other reasons may be that family members consented only for the procurement of lungs and not hearts and concerns about ischemic damage to hearts from donors after cardiac death [19]. Unless administrative data coding is improved, accurate information about organ-specific donations will require alternative strategies, for example manual chart review and data collection.

From our data sources we could not understand why the codes resulted in limited sensitivity. However, some surgeons may receive funding from alternative payment programs and may not always bill for these codes. Further, we spoke to a few billing clerks and found that it may have been possible that surgeons billed the recipients rather the donor for these services. We were surprised that the transplant counseling performed very poorly at identifying deceased organ donors. The purpose of this service is to provide a potential donor's family members with sufficient information and clinical data to enable them to make an informed decision regarding organ donation and transplantation. However, in recent years, organ procurement organizations began employing designated transplant coordinators responsible for obtaining consent for donation from family members. As a result, physicians may play a smaller role in transplant counseling and therefore do not bill for this particular service.

Future research should examine other central elements in the deceased organ donor process. As a priority, algorithms should be developed and validated to identify all deaths eligible for deceased organ donation in large administrative databases. This project would require manual chart abstractions of all those who were deemed eligible and approached for organ donation to serve as the reference standard. However, such project may not be feasible with Ontario's administrative data based on our experience with this study. We will not be able to use any diagnosis or procedural hospital codes that are associated with being an eligible donor because 15% of our reference standard donors were missing when we restricted our cohort to hospitalized deaths. Further, in our pilot work, we were not able to identify any billing code associated with mechanical ventilation that was common among all donors.

Many countries hoping to improve organ donation will require information on their own deceased donors. We recommend that primary data collection (i.e. chart abstraction) be conducted to collect accurate information on deceased donors, particularly organ-specific donors. If primary data collection is too costly or timely, decision-makers may consider the use of physician claims to identify deceased donors while recognizing its limitations. Other alternative approaches to identify deceased organ donors beyond the use of chart abstraction and physician claims include the use of national donor registries. Certain countries may have national donor registries that could be cross-linked to other databases. However, some national registries do not collect or distribute patient identifiers that are needed to link clinical or claims data [20]. An advantage of using administrative data is that it will provide similar information on both those who did and did not become donors (the latter is not recorded in donor registries).

Our study has several strengths. We conducted manual, comprehensive reviews of the medical charts of all deceased donors in our province for the period of interest to create a reference standard. This approach served to minimize the risk of misclassifying organ donors in our reference standard. Our algorithm utilized existing, population-based databases that can be used to study patterns of organ donation across entire regions. Ours is the first study to demonstrate that identifying deceased organ donors using billing algorithms in administrative data has certain limitations. Our experience will help guide the way for future system-level research in Ontario. For example, a recent study used our codes to compare donor rates between transplant hospitals and large general hospitals from 1994 to 2011 [21].

There are several limitations to our study. We used a simple and pragmatic approach to identify organ donors using administrative data, and future research is required to further refine methods for identifying organ-specific donations. Although our algorithm displayed reasonable performance in correctly identifying deceased organ donors, improvements in the performance of the algorithm may be possible using alternative approaches. For example, future work can use machine learning techniques to take information from large administrative databases in an automated fashion to compile the most efficient algorithms [22]. Further, neurological determination of death donors were the most popular donors during our study period. The popularization of donors after cardiac deaths may require different algorithms. In addition, we were not able to identify organs that were procured but not transplanted. Finally, our algorithm used a physician billing database that is unique to Ontario and thus the accuracy of the algorithm may differ in other regions. This limitation applies to all studies examining the validity of case-finding codes in administrative databases [7], [11], [23]–[27]. Similar to other validation studies, the translation of this algorithm to comparable codes in other regions would need to be verified [9]. Our findings are likely of interest to other healthcare systems similar to Ontario where fee-for-service billing is the primary method of physician reimbursement.

In conclusion, our findings reveal that researchers should use primary data abstraction to identify deceased organ donors in large healthcare databases. In healthcare systems where a fee-for-service billing is the primary method of physician reimbursement, researchers may consider using physician claims data to identify deceased organ donors if such data does not exist or is not feasible to collect. However, physician claims may result in missed unidentified cases of organ donation. Further, using physician claims data to identify organ-specific donors (e.g. heart donors) will require improved coding and is not recommended. Our best-performing algorithm for identifying organ donation after death consisted of a combination of heart, lung, liver and kidney procurement billing codes. When primary data are not available, this algorithm can be used in future research to examine trends in deceased organ donation, to evaluate regional disparities in organ donation, and to explore factors that influence its likelihood.

Supporting Information

Acknowledgments

Disclaimer: The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred.