Conceived and designed the experiments: JMF WSG AJJAS RPK. Performed the experiments: JMF RPK. Analyzed the data: JMF AMM. Contributed reagents/materials/analysis tools: JMF AMM. Wrote the paper: JMF WSG AMM AJJAS RPK.
The authors have declared that no competing interests exist.
Incident reporting systems (IRS) are used to identify medical errors in order to learn from mistakes and improve patient safety in hospitals. However, IRS contain only a small fraction of occurring incidents. A more comprehensive overview of medical error in hospitals may be obtained by combining information from multiple sources. The WHO has developed the International Classification for Patient Safety (ICPS) in order to enable comparison of incident reports from different sources and institutions.
The aim of this paper was to provide a more comprehensive overview of medical error in hospitals using a combination of different information sources. Incident reports collected from IRS, patient complaints and retrospective chart review in an academic acute care hospital were classified using the ICPS. The main outcome measures were distribution of incidents over the thirteen categories of the ICPS classifier “Incident type”, described as odds ratios (OR) and proportional similarity indices (PSI).
A total of 1012 incidents resulted in 1282 classified items. Large differences between data from IRS and patient complaints (PSI = 0.32) and from IRS and retrospective chart review (PSI = 0.31) were mainly attributable to behaviour (OR = 6.08), clinical administration (OR = 5.14), clinical process (OR = 6.73) and resources (OR = 2.06).
IRS do not capture all incidents in hospitals and should be combined with complementary information about diagnostic error and delayed treatment from patient complaints and retrospective chart review. Since incidents that are not recorded in IRS do not lead to remedial and preventive action in response to IRS reports, healthcare centres that have access to different incident detection methods should harness information from all sources to improve patient safety.
It has been increasingly recognised that hospitals can be dangerous places for patients, since medical errors have been shown to cause harm to patients
IRS are not the only source of information for studies of incidents in hospitals. Thomas et al. described eight methods of detecting errors and adverse events, including chart review, malpractice claims analysis, observation of patient care, and IRS
In the absence of a universally accepted incident classification system, studies focusing on IRS have used different classification systems
For the present study we retrieved information from different sources (incident reports, patient complaints and retrospective chart review of deceased patients) to identify incidents and adverse events. We classified this combined information using the ICPS in order to create a comprehensive picture of incidents occurring in hospitals. We specifically addressed the following research question: Are the different information sources complementary with regard to the types of incidents they report?
In accordance with Dutch Law on Medical Scientific Research, retrospective research using patient charts was automatically granted ethics approval in the participating institutions and there was no requirement for individual patient consent, provided confidentiality was maintained.
We collected data from a medium sized (700 beds) academic acute care hospital in the Netherlands, serving both adults and paediatric patients. Three information sources were used: 1) all incident reports for 2007; 2) patient complaints filed in 2007; 3) retrospective chart reviews of all inpatients that died in 2008. These information sources applied to different subgroups able to provide information about adverse events in hospitals. We used data for 2007 to ensure that incidents could not be traced back to staff or patients and referred to events before the introduction of statutory safety management systems in 2008.
Because of anonymity of patient and staff information, overlap between incidents from different sources could not be detected. The results are therefore not presented as absolute differences between information sources, but as distributions of incidents over categories.
Incidents were reported on paper. All hospital personnel are authorised to report incidents, and the IRS contains information about nature, severity and place of incidents and about action taken to prevent recurrence. We transformed the available data for 2007 into a digital data file.
Any patient can file a complaint against the hospital or an individual healthcare provider. We collected all written patient complaints, handled in 2007 by a complaint mediator or the complaints committee. Complaints not directly related to patient care (such as complaints about billing) were not included in the study (N = 59).
The hospital has a committee, consisting of six medical doctors and seven nurses, which retrospectively inspects the files of all deceased patients in order to identify any adverse events. The review method and definitions are based on similar national research
We use the term ‘reports’ to refer to incidents from the IRS, from patient complaints and from retrospective chart review.
We classified all reports as ‘incident type’. This ICPS classifier, which contains thirteen categories (
Category | Example | |
|
behaviour | treatment of patient by staff was inconsiderate or rude |
|
blood/ blood products | request for a blood product was for the wrong patient; or blood with the wrong blood type was administered to a patient |
|
clinical administration | wrong documents were filled out for admission; or a patient was treated by different doctor than previously discussed |
|
clinical process/ procedure | a delay in treatment due to postponement of surgery; or a diagnosis was missed |
|
documentation | patient chart was missing; or information on patient chart was incorrect or missing |
|
health care ass. infection | patient develops infection near the surgical site, due to a gauze that has been left behind in the wound. |
|
infrastructure | trolley does not fit into the lift; or nurse slips on wet floor |
|
medical device/ equipment | computer malfunction or surgical tools that break or are unsterile |
|
medication/iv fluids | wrong drug is administered to the patient; or patient has not received medication |
|
nutrition | wrong quantity or wrong sort of drip-feed is administered |
|
oxygen/gas/vapour | patient returns from procedure and a nurse forgets to connect the oxygen |
|
patient accidents | patient that has fallen out of bed; or patient that has fallen in the bathroom |
|
resources/organizational management | understaffing or no available beds |
A report can fall into several categories
JMF classified a sample (from all three sources) of 300 reports and discussed the results with a second researcher (RPK) until consensus was reached. JMF then classified the remaining reports, while a random sample of 10% was also classified by RPK in order to determine interrater reliability using Cohen's kappa. Since Cohen's kappa is based on the assumption that one item cannot be in more than one category, only the first classification of each report, representing the main category for that report, was used to calculate kappa. Kappa was 0.73, indicating substantial interrater agreement
There are several reasons why we deemed the ICPS suitable for our study: 1) It was developed using a Delphi procedure
We calculated the proportional similarity index (PSI) for distributions of the relative frequencies of incident reports from two information sources over ICPS categories in order to determine whether the sources were complementary
We calculated odds ratios to determine if a specific ICPS category was more likely to be present in the IRS or in one of the two other information sources. A high odds ratio (OR≥2) indicates that an incident of this category is more frequently represented in either of the other information sources (patient complaints or retrospective chart review) than in the IRS. A low odds ratio (OR≤1) shows that an incident of this category was more strongly represented in the IRS. SPSS 15.0 was used for all calculations.
The number of reports from each information source and the total number of classified items (including 2nd and 3rd categories for some incidents) are displayed in
Information source | Number of incidents (N) | Total number of classified items (incl. 2nd and 3rd category) (N) |
Incident reports | 736 | 904 |
Patient complaints | 235 | 327 |
Retrospective chart review | 44 | 51 |
Total | 1015 | 1282 |
A: subcategorie of “Behaviour”. B: subcategorie of “Clinial administration”. C: subcategorie of “Clinical Process”. D: subcategorie of “Resources/ organizational management”.
The primary aim of this study was to investigate whether information about reported incidents differed between information sources. The distribution of reports over categories and subcategories of the ICPS class ‘Incident Type’ showed remarkable differences between incident reports, patient complaints and retrospective chart review of deceased patients. This suggests that a combination of detection methods, using information from patients
Patient complaints differed from IRS in several ways. First of all, patient complaints revealed more incidents in the category clinical process, particularly in relation to diagnosis, general care and procedure/treatment. Particularly striking is the difference between patient complaints and IRS in diagnosis-related incidents, mostly relating to delay in diagnosis or wrong or missed diagnoses. This is surprising, as one would expect healthcare workers to be aware of and therefore report missed diagnoses. The literature reports extensively on the prevalence of diagnostic errors and their impact on patient safety
Secondly, patient complaints identified more incidents in the category behaviour, inconsiderate behaviour in particular. Previous research has shown that inconsiderate behaviour or unprofessional conduct is one of the main reasons for patient complaints or lawsuits
Thirdly, patient complaints revealed more incidents in the category clinical administration in relation to waiting lists, management of appointments and task allocation, such as complaints about being seen or operated upon by a different doctor than expected or agreed upon. Complaints about waiting lists and management reports have also been reported elsewhere
Apart from patient complaints we gathered incident reports from retrospective chart review, which is generally considered the gold standard measurement of incidents occurring in hospitals
This study has several limitations. Firstly, most of the data were collected in one academic medical centre. Consequently, the results may not be generalisable to other hospitals or other countries. Secondly, because of anonymity of patient and staff information, overlap between incidents from different sources could not be detected. This might result in a slight overestimation of some incident types. Thirdly, we used ICPS to classify incidents in order to improve the comparability of findings. However, the ICPS is still under development and needs to be tested with more and different databases of other healthcare centres in order to optimise the (sub)categories.
There are also several practical implications to this study. First of all, the results suggest that IRS alone does not provide a comprehensive picture of what goes wrong in a hospital. Moreover, the fact that diagnostic errors and delay in treatment are rarely reported in IRS impacts on actions undertaken to remedy and prevent such incidents. Healthcare centres using more than one method of incident detection (e.g. methods relying on patients and health care workers as sources of information) should combine these data, preferably using the same classification for each source, in order to enhance comparability. This will give a better insight into the most prevalent latent and active errors, and can help to prioritise which of these problems should receive immediate attention and which are less urgent.
The second practical implication considers its use for medical education. The incidents that were identified can be used to educate medical students, residents and faculty about patient safety issues. Incidents can enhance awareness of vulnerabilities of hospital organisations and identify which situations are more conducive to error. Increased attention through education could increase doctors' awareness of these situations and, consequently, reduce the number of (e.g. diagnostic) errors. We therefore recommend that medical schools should incorporate this information in their courses on patient safety.
(DOC)
The authors would like to thank Mereke Gorsira for editing the final version of the manuscript.