Figures
Abstract
For research findings to be generalized, a sample must be representative of the actual population of interest. Lower limb amputation is most frequently performed in older patients with vascular disease, a population that is often under-represented in research. The aim of this study was to explore the impact of selection bias by comparing characteristics from a sample included in a prospective study of phantom pain with the actual population who underwent amputation. Only 27% of all potential patients were referred during the first year of the prospective study. The referred patients were 8 years younger (p<0.001) and less likely to have had amputation because of a vascular condition, diabetes or infection (p = 0.003) than those not referred. There was also a significant difference in one year survival between the groups; 67% of referred patients survived compared with just 40% of non-referred patients (p = 0.004). The biased population in the phantom pain study may have resulted in an underestimation of phantom pain in the original study and subsequent protective factors should be considered within the context of the younger population reported. Selection bias is common in amputation research, and research methods to minimize its impact must be given greater attention.
Citation: Fortington LV, Geertzen JHB, Bosmans JC, Dijkstra PU (2012) Bias in Amputation Research; Impact of Subjects Missed from a Prospective Study. PLoS ONE 7(8): e43629. https://doi.org/10.1371/journal.pone.0043629
Editor: Giuseppe Biondi-Zoccai, Sapienza University of Rome, Italy
Received: February 7, 2012; Accepted: July 26, 2012; Published: August 20, 2012
Copyright: © Fortington et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was funded in part by grants provided by the OIM Foundation, Assen and the Beatrixoord Foundation North Netherlands, Haren. No additional external funding received for this study. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
After a lower limb amputation (LLA), people face a number of challenges including loss of mobility, altered body image and phantom pain. Research to better understand these consequences allows planning for rehabilitation and long term care and builds an evidence base from which we can more accurately inform patients on their expected outcomes. However, difficulties with population sampling are frequent in amputation research and this impacts our ability to draw accurate conclusions.
LLA is most frequently performed in older patients with vascular disease, a population that is, for the most part, under-represented in research [1]. Multiple co-morbidities and cognitive decline can prevent this sub-population from meeting required inclusion criteria. The issue of bias in studies of elderly people is well recognized [1]–[4]. The same is true of amputation research, with authors’ invariably describing selection bias within their sample as a limitation of their study [5]–[11]. However, the impact of this bias is rarely described [12]. It is important to understand this research limitation in applying results to clinical practice and to better design future studies.
In a prospective study of phantom limb pain, it was noted that the population characteristics of the included sample were considerably different from what would be expected in the LLA population [5]. The aim of the current study is to explore the impact of this bias on the primary outcomes (factors associated with phantom pain) by comparing the study sample with the actual population who underwent amputation.
Methods
Ethics Statement
The study protocol was approved by the medical ethics committees of the University Medical Center Groningen. Patients participating in the phantom pain study provided their written informed consent.
Setting
Both studies were conducted in the 3 Northern provinces of the Netherlands: Groningen, Friesland and Drenthe. Fourteen hospitals in the region performed lower limb amputations, generally under the care of a vascular surgeon. This study looks at patients who had a first ever unilateral transtibial amputation, knee disarticulation or transfemoral amputation between 1 January 2004 and 31 December 2004.
Phantom Pain (PP) Study
A prospective study ran from 1 November 2003 to 30 April 2008. At a face-to-face meeting, and confirmed afterwards in writing, surgeons were informed about the study including the aims, inclusion and exclusion criteria, and recruitment procedures.
The surgeons were requested to include all patients: (a) aged ≥18 years; (b) undergoing primary major amputation (at or proximal to metatarsophalangeal level); and (c) able to read and write in Dutch. The primary investigator discussed the study aims with the patient and they were asked to participate and give their written consent.
Exclusion criteria were: (a) had a previous ipsilateral amputation; (b) were too unwell or showed signs of clinical dementia which prevented completion of the questionnaires, or (c) were recruited more than 5 days after the amputation.
If the surgeons themselves decided to exclude a patient, they agreed to send the characteristics of the patient (age, sex) and amputation details (level, cause) to the primary investigator to ensure a complete census of patients was recorded. The primary investigator maintained regular contact with the study coordinator at each hospital.
Population Study
In 2010, surgeons from each hospital were contacted about a new study on the incidence of LLA. Surgeons from all hospitals agreed to participate. They were requested to compile a list of patients who underwent major amputation in 2004. The medical records of these patients were reviewed between August 2010 and July 2011 for patient data (age, sex), amputation details (level, cause), marital status, comorbidities and medical history including previous minor amputations or peripheral vascular procedures (angioplasty, embolectomy or peripheral bypass) and survival or date of death. To ensure a complete survival dataset, general practitioners were contacted for patients whose status was not up to date in hospital records.
Statistical Analysis
Characteristics of referred patients (irrespective of whether they were included or excluded from analyses in the original study) were compared with the non-referred patients using chi-square tests for categorical variables, Mann-Whitney U test for age distribution and log rank tests for survival. Significance was set at 0.05 and analyses were performed using PASW Statistics version 18.0.
Results
Surgeons representing 12 of 14 hospitals attended the information meeting of the PP study. Two hospitals were unable to participate because of restrictions from their local administration and medical ethics procedures. Surgeons from ten hospitals agreed to participate in recruitment of patients.
From the current study, one hospital was unable to identify the relevant files because of changes in their database. This hospital was excluded and subsequently, one patient from this hospital who had been referred to the PP study was excluded.
Thirty nine (27%) of a possible 146 patients were referred during the first full year of the study (table 1). The referred population had a median age 8 years younger (p<0.001) and were less likely to have had amputation because of a vascular condition, diabetes or infection (p = 0.003) than those who were not referred. More non-referred patients had bilateral amputation while more referred patients had a knee disarticulation (p = 0.049). No differences in the number or type of major co-morbidities were seen, although referred patients were more likely to have had undergone a previous vascular intervention (p = 0.042) such as a peripheral bypass procedure or angioplasty. Referred patients were more frequently discharged home or to a rehabilitation centre with non-referred patients more often discharged to a care centre (p = 0.020).
There was a significant difference in one year survival between the groups; 67% of referred patients survived compared with 40% of non-referred patients (p = 0.004). Overall survival time after amputation also differed significantly (figure 1): median (standard error) survival for referred group = 41.1 (7.9) months, non-referred = 13.6 (6.6) months, χ2(1df) = 5.6; p = 0.018.
To verify whether or not the differences in the groups were linked to the significantly poorer survival of the non-referred population, characteristics of patients who survived to 12 months are presented in table 2. There remained a significantly younger median (p = 0.016) and mean (p = 0.007) age difference between the referred and non-referred group. Again, the referred group were more likely to have had a knee disarticulation, less likely to have had a transtibial amputation (p = 0.041), and more frequently had amputation because of non-vascular causes (0.012). There were no other significant differences between the referred and non-referred groups in 12 month survivors. Overall survival differed by 9 months (median (standard error) survival for referred group = 64.1 (14.7) months, non-referred = 55.6 (11.5) months, χ2(1df) = 1.8; p = 0.177) and non-referred patients were more frequently discharged to a care center while referred patients were more often discharged to a rehabilitation center (p = 0.130).
Discussion
A prospective study of phantom pain aimed to report all patients undergoing primary major lower limb amputation yet more than 70% of potential participants were not referred in the first year. This resulted in a sample that was younger, less likely to have had vascular related amputation and differed in both pre and post care setting than the actual population who underwent LLA [5].
The PP study described a prevalence rate for phantom pain of 32% measured 6 months after amputation [5]. Other literature measuring occurrence at 6 months has reported more than double this amount, with 65–79% of people having phantom pain [13], [14]. Occurrence rates at 6 months in trials to treat phantom pain range from 0–38% (0/10 in intervention; 5/13 in control group) [15] to 9–73% (1/11 in intervention; 8/11 in control) [16]. The PP study has a lower rate of phantom pain than expected which raises some uncertainty in generalisation and clinical application of the protective factors identified. These protective factors should be considered within the context of the biased population.
Three protective factors against the development of phantom pain were described: being male, having a lower limb amputation (versus an upper limb amputation) and time since amputation [5]. In the current study, there were no differences in sex between referred and non-referred patients (upper limb amputations were not included and only one year of the PP study was analysed). Other factors in the PP study were also investigated but not found to be significant, including level of amputation, cause of amputation and age at time of amputation. None of these factors were accurately represented by the sample referred to the PP study and it is not possible to draw a valid conclusion over their influence based only on this data.
The factors identified in the PP study were largely in disagreement to other literature. In addition to prevalence rates being much higher than the PP study, sex is reported as being unrelated to occurrence of phantom pain [13], [17] although males and females may deal with the pain differently [18]. Increasing age is shown as having a higher risk of phantom pain [19] while others have reported no relation [17] or not included age in their analysis [13]. More proximal amputation levels, and having bilateral amputation, may increase a person’s risk of phantom pain [19] although again others have found no association between the two [13], [20].
The contradictory findings surrounding phantom pain in these different populations are, at least in part, also partly attributable to differences in definitions and study design. Cut-off points for what constitutes phantom pain can include people with almost constant pain or people who experience only occasional pain [5]. Most previous studies of phantom pain are cross sectional and direct cause and effect cannot be stated. Studies include people with differing lengths of time since amputation, from a few months to many years [18], [19], [21], yet time since amputation is another factor potentially linked to phantom pain. Poor physical condition from co-morbidities and cognitive deficit leads to difficulties in patient inclusion and sample sizes are generally small. As amputation research is also characterised by a high mortality rate, follow up rates are often low. In this study almost 50% of the total population had died within 12 months of their first major amputation, including 33% of the referred group. The PP study is the largest longitudinal study of phantom pain performed (total included at first follow up was 85 from 120 included) and followed patients for up to 3.5 years [5]. Unfortunately, the substantial bias seen in the population presents a major limitation and there remains limited evidence around risk factors associated with phantom pain. Reviews looking at mechanisms and treatment of phantom limb pain reveal similar shortcomings in methodology [22]–[26].
A major difficulty with amputation outcome research is obtaining large and representative samples. The reasons for having an amputation make it difficult for many cases to be included in research as elderly people with systemic disease tend not to be considered for participation and have a higher rate of drop out or death [1]. This appeared to be a key element of (non)recruitment to the PP study, with patients who were older and with amputation due to vascular disease least likely to be referred. Data, or at least their estimates, on non-participants (including people who did not give consent, patients who are excluded, deaths and drop outs) should be communicated by authors. In the PP study, all referral sources were requested to provide this information, but unfortunately it did not occur. Minimal data presented in amputation research should include the number of participants and non-participants, age, sex, level of amputation and cause of amputation.
Our data were split to look at 12 month ‘non-survivors’ compared to ‘survivors’. With outcomes of interest for the ‘frailer’ group likely differing from the survivors, it is reasonable that they are not included in longitudinal outcome research. Unfortunately, our results showed that a substantial number of this healthier group, the 12-month survivors, also failed to be included in the PP study. In designing any study, gaining strong interest and support from relevant stakeholders and referral sources is vital. In the case of the PP study, referral sources (surgeons and staff) were informed of the aims and methodology at a regional meeting, with verbal agreements given for participation (referral of patients). The high referral rate (>85% of all within hospital, contributing >69% of all referred) from the study’s operating/base hospital, suggests either the physical presence of the investigator and/or simply being the study’s main location are the most effective strategies for recruitment. Across the entire regional network of hospitals, a physical presence was not possible. Attempts to counter this limitation through regular phone and newsletter contact were unsuccessful, with 6 hospitals not referring patients in the first year despite their agreement. Improving recruitment via clinicians is a difficult task; even large randomised controlled trials have great difficulty identifying successful strategies [27]. Adding to this are strict timeframes of the inclusion criteria of the PP study with referral set for within 5 days. This meant that surgeons (and their staff) were primarily responsible for identifying cases, at a time when other factors, such as pre-operative assessment, can naturally be of a greater priority. It is not routine practice for the rehabilitation physician to be involved pre-surgically so this additional referral source was not utilised. Other alternatives were not considered as clinically relevant options, such as extending the inclusion period to >5 days, as this would have introduced problems with recall.
Limitations
The population study used data from a retrospective review of medical files and as such, information was limited to what is included in these. The data were collected for a concurrent study on incidence and as such, they are considered to be complete. However, we acknowledge that cases may have been missed. If anything, the sample is an underestimation, although we do not expect that this would have any large affect on our main findings. Another limitation from the study design (review of medical records) is not having access to information on disease severity or duration of disease. Further, unless it is of a very severe nature, cognitive status is infrequently noted in the medical files. However, this is likely to be a major source of selection bias in LLA research given that vascular disease affects the body systemically. Finally, there was no information on survival status available for 27 patients and our results are likely to be an underestimate of survival time.
In the current study, all patients referred to the PP study were considered as one group. However, 16 (41%) of these patients were not part of the analyses as they did not meet the criteria for inclusion. These excluded patients were older and more likely to have had amputation because of vascular disease than included patients [5]. The findings of this current study should therefore be considered as a conservative estimate of the impact of selection bias as these excluded older patients with vascular disease remained within the ‘referred’ group.
Conclusion
Selection bias is common, and perhaps inherent, in amputation research. Over 70% of patients were missed in a study of phantom pain, resulting in a younger population who were less likely to have had vascular related amputation and differed in respect to their pre- and post-care setting. As a result, phantom pain was possibly underestimated and the resultant protective factors identified should be considered only within the context of the biased population. Two important elements for improving research into amputation outcomes were identified: (a) failure to refer relevant cases (recruitment bias); and (b) failure to communicate reasonable non-inclusions. Potential bias should be more clearly presented by authors and subsequent conclusions and clinical decisions made with greater caution. In addition, maximum efforts should be directed to research methodology which minimises the influence of bias.
Acknowledgments
For their active involvement in the studies, we thank the patients, surgeons and colleagues from all participating hospitals in Groningen, Drenthe and Friesland. We also thank Ms A Teunissen, Mr K Esselink (student assistants), Dr GM Rommers and Prof dr K Postema for their assistance with data collection.
Author Contributions
Conceived and designed the experiments: LVF JHBG JCB PUD. Performed the experiments: LVF JCB. Analyzed the data: LVF PUD. Wrote the paper: LVF JHBG JCB PUD.
References
- 1. Hardy SE, Allore H, Studenski SA (2009) Missing data: A special challenge in aging research. J Am Geriatr Soc 57: 722–729.
- 2. Chatfield MD, Brayne CE, Matthews FE (2005) A systematic literature review of attrition between waves in longitudinal studies in the elderly shows a consistent pattern of dropout between differing studies. J Clin Epidemiol 58: 13–19.
- 3. Euser SM, Schram MT, Hofman A, Westendorp RG, Breteler MM (2008) Measuring cognitive function with age: The influence of selection by health and survival. Epidemiology 19: 440–447.
- 4. Hernan MA, Alonso A, Logroscino G (2008) Cigarette smoking and dementia: Potential selection bias in the elderly. Epidemiology 19: 448–450.
- 5. Bosmans JC, Geertzen JH, Post WJ, van der Schans CP, Dijkstra PU (2010) Factors associated with phantom limb pain: A 31/2-year prospective study. Clin Rehabil 24: 444–453.
- 6. O’Neill BF, Evans JJ (2009) Memory and executive function predict mobility rehabilitation outcome after lower-limb amputation. Disabil Rehabil 31: 1083–1091.
- 7. Unwin J, Kacperek L, Clarke C (2009) A prospective study of positive adjustment to lower limb amputation. Clin Rehabil 23: 1044–1050.
- 8. Geertzen JHB, Bosmans JC, van der Schans CP, Dijkstra PU (2005) Claimed walking distance of lower limb amputees. Disabil Rehabil 27: 101–104.
- 9. van der Schans CP, Geertzen JHB, Schoppen T, Dijkstra PU (2002) Phantom pain and health-related quality of life in lower limb amputees. J Pain Symptom Manage 24: 429–436.
- 10. Miller WC, Speechley M, Deathe AB (2002) Balance confidence among people with lower-limb amputations. Phys Ther 82: 856–865.
- 11. Ehde DM, Czerniecki JM, Smith DG, Campbell KM, Edwards WT, et al. (2000) Chronic phantom sensations, phantom pain, residual limb pain, and other regional pain after lower limb amputation. Arch Phys Med Rehabil 81: 1039–1044.
- 12. Bosmans JC, Geertzen JH, Hoekstra HJ, Dijkstra PU (2010) Survival of participating and nonparticipating limb amputees in prospective study: Consequences for research. J Rehabil Res Dev 47: 457–464.
- 13. Jensen TS, Krebs B, Nielsen J, Rasmussen P (1985) Immediate and long-term phantom limb pain in amputees: Incidence, clinical characteristics and relationship to pre-amputation limb pain. Pain 21: 267–278.
- 14. Richardson C, Glenn S, Nurmikko T, Horgan M (2006) Incidence of phantom phenomena including phantom limb pain 6 months after major lower limb amputation in patients with peripheral vascular disease. Clin J Pain 22: 353–358.
- 15. Bach S, Noreng MF, Tjellden NU (1988) Phantom limb pain in amputees during the first 12 months following limb amputation, after preoperative lumbar epidural blockade. Pain 33: 297–301.
- 16. Jahangiri M, Jayatunga AP, Bradley JW, Dark CH (1994) Prevention of phantom pain after major lower limb amputation by epidural infusion of diamorphine, clonidine and bupivacaine. Ann R Coll Surg Engl 76: 324–326.
- 17. Nikolajsen L, Jensen TS (2001) Phantom limb pain. Br J Anaesth 87: 107–116.
- 18. Hirsh AT, Dillworth TM, Ehde DM, Jensen MP (2010) Sex differences in pain and psychological functioning in persons with limb loss. J Pain 11: 79–86.
- 19. Dijkstra PU, Geertzen JHB, Stewart R, Van DS (2002) Phantom pain and risk factors: A multivariate analysis. J Pain Symptom Manage 24: 578–585.
- 20. Rayegani SM, Aryanmehr A, Soroosh MR, Baghbani M (2010) Phantom pain, phantom sensation, and spine pain in bilateral lower limb amputees: Results of a national survey of Iraq-Iran war victims’ health status. JPO 22: 162–165.
- 21. Byrne KP (2011) Survey of phantom limb pain, phantom sensation and stump pain in Cambodian and New Zealand amputees. Pain Med 12: 794–798.
- 22. Wolff A, Vanduynhoven E, van Kleef M, Huygen F, Pope JE, et al. (2011) 21 Phantom pain. Pain Pract 11: 403–413.
- 23. Halbert J, Crotty M, Cameron ID (2002) Evidence for the optimal management of acute and chronic phantom pain: A systematic review. Clin J Pain 18: 84–92.
- 24. Flor H (2002) Phantom-limb pain: Characteristics, causes, and treatment. Lancet Neurol 1: 182–189.
- 25. Subedi B, Grossberg GT (2011) Phantom limb pain: Mechanisms and treatment approaches. Pain Res Treat 2011: 864605.
- 26. Casale R, Ceccherelli F, Labeeb AA, Biella GE (2009) Phantom limb pain relief by contralateral myofascial injection with local anaesthetic in a placebo-controlled study: Preliminary results. J Rehabil Med 41: 418–422.
- 27. Fletcher B, Gheorghe A, Moore D, Wilson S, Damery S (2012) Improving the recruitment activity of clinicians in randomised controlled trials: A systematic review. BMJ Open 2: e000496.