Studies of the genetic basis of drug response could help clarify mechanisms of drug action/metabolism, and facilitate development of genotype-based predictive tests of efficacy or toxicity (pharmacogenetics).
We conducted a systematic review and field synopsis of pharmacogenetic studies to quantify the scope and quality of available evidence in this field in order to inform future research.
Original research articles were identified in Medline, reference lists from 24 meta-analyses/systematic reviews/review articles and U.S. Food and Drug Administration website of approved pharmacogenetic tests.
Study Eligibility Criteria, Participants, and Intervention Criteria
We included any study in which either intended or adverse response to drug therapy was examined in relation to genetic variation in the germline or cancer cells in humans.
Study Appraisal and Synthesis Methods
Study characteristics and data reported in abstracts were recorded. We further analysed full text from a random 10% subset of articles spanning the different subclasses of study.
From 102,264 Medline hits and 1,641 articles from other sources, we identified 1,668 primary research articles (1987 to 2007, inclusive). A high proportion of remaining articles were reviews/commentaries (ratio of reviews to primary research approximately 25:1). The majority of studies (81.8%) were set in Europe and North America focussing on cancer, cardiovascular disease and neurology/psychiatry. There was predominantly a candidate gene approach using common alleles, which despite small sample sizes (median 93 [IQR 40–222]) with no trend to an increase over time, generated a high proportion (74.5%) of nominally significant (p<0.05) reported associations suggesting the possibility of significance-chasing bias. Despite 136 examples of gene/drug interventions being the subject of ≥4 studies, only 31 meta-analyses were identified. The majority (69.4%) of end-points were continuous and likely surrogate rather than hard (binary) clinical end-points.
Conclusions and Implications of Key Findings
The high expectation but limited translation of pharmacogenetic research thus far may be explained by the preponderance of reviews over primary research, small sample sizes, a mainly candidate gene approach, surrogate markers, an excess of nominally positive to truly positive associations and paucity of meta-analyses. Recommendations based on these findings should inform future study design to help realise the goal of personalised medicines.
Systematic Review Registration Number
Citation: Holmes MV, Shah T, Vickery C, Smeeth L, Hingorani AD, et al. (2009) Fulfilling the Promise of Personalized Medicine? Systematic Review and Field Synopsis of Pharmacogenetic Studies. PLoS ONE 4(12): e7960. doi:10.1371/journal.pone.0007960
Editor: Yuan Luo, University of Maryland School of Pharmacy, United States of America
Received: July 31, 2009; Accepted: October 23, 2009; Published: December 2, 2009
Copyright: © 2009 Holmes et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The work reported was not the subject of a specific research grant. Aroon D. Hingorani is funded by a Senior Research Fellowship from the British Heart Foundation (FS 05/125). Liam Smeeth is funded by a Wellcome Trust Senior Research Fellowship in Clinical Science (082178). Michael V. Holmes is funded by an Academic Clinical Fellowship from the National Institute for Health Research and a Population Health Scientist Fellowship from the Medical Research Council (G0802432). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: Aroon Hingorani is on the Editorial Board of the Drug and Therapeutics Bulletin (International Society of Drug Bulletins). He has received honoraria for speaking at educational meetings with a pharmaceutical sponsor but has donated these in whole or part to various medical charities. He has acted as a consultant to London Genetics and to GSK.
Individual differences in drug efficacy, or susceptibility to adverse effects, collectively make an important contribution to the burden of ill-health , . Studying the genetic basis could reduce this by clarifying pathways and mechanisms of drug action or metabolism to inform drug development, and by the development of genotype-based predictive tests of efficacy or toxicity (pharmacogenetics).
As with research in common disease susceptibility, the path to translation involves a two stage process that first requires the reliable identification of the genetic loci involved, and then research into the healthcare applications of this knowledge, which includes critical appraisal of the performance of genotype as a predictive test. While the extent of the clinical impact of research in both areas is uncertain, the reliable identification of loci involved in drug response (pharmacogenetics) appears to be less advanced than the identification of susceptibility loci for common disease . After more than two decades of research, a continuing expansion in the range and depth of available drug therapies, and the continued promise of ‘personalized medicine,’, , , , ,  only four pharmacogenetic tests were mandated as part of the FDA drug approval pre-July 2009,  while for another 10 tests recommended by the FDA, clinical utility is not universally agreed , , . Understanding the reasons for the blocks in development of personalised medicines could help improve efficiency of future research.
Systematic reviews and field synopses previously exposed the obstacles to progress in complex disease genetics. These included: a focus on candidate genes rather than genome-wide analysis; inadequate sample size; suboptimal capture of genetic variation; and significance chasing and reporting bias; all of which led to a failure to replicate and validate genetic associations , , . These overviews , ,  were followed by improvements in research design which made an important contribution to the recent success in the identification in genes for common disease . These considerations and the absence of a prior systematic, quantitative overview of pharmacogenetic research was the motivation for the current study.
We followed PRISMA 2009 guidelines .
We identified pharmacogenetic studies using a carefully designed search strategy. We searched articles indexed in Medline using the Medical Subject Heading (MeSH) or full text terms (“Genetic Variation”[MeSH] or “Genotype”[MeSH] or “Genes”[MeSH] or genotype* or polymorphism* or allele* or mutation*) and (“Treatment Outcome”[MeSH] or “Therapeutics”[MeSH] or “adverse effects”[Subheading] or “Pharmacogenetics”[MeSH] or “Toxicogenetics”[MeSH] or pharmacogenomic* or pharmacogenetic* or toxicogenetic* or therapeutic* or intervention* or treatment*) from inception up to 01-01-2008. The search was initially restricted to Human studies and subsequently to Clinical Trials, Meta-Analyses, Practice Guidelines, and Randomized Controlled Trials using the Medline filters and by doing so excluded Editorials, Reviews and Letters. We supplemented the search with relevant references indexed in 12 meta-analyses and 12 review articles (spanning most disease categories). The FDA “Table of Valid Genomic Biomarkers in the Context of Approved Drug Labels” was also cross-referenced and to identify potentially missing meta-analyses, the ten most frequently studied genes in each category (germ-line [kinetic/dynamic] and somatic) were individually searched in Medline (none extra was found). Furthermore, as some meta-analyses are indexed in Medline as reviews and thus had the potential to be excluded during the initial search, we repeated our Medline search selecting only meta-analyses.
To be eligible for inclusion, studies had to satisfy our definition of a pharmacogenetic study: a study in which the response (intended outcome/adverse reaction) to drug therapy was examined in relation to genetic variation (germline/somatic) in humans. It was mandatory that participants be genotyped (studies using phenotype as a surrogate of genetic variation were excluded) and that >1 allelic variations at a gene were analysed (in order to compare differing alleles on response to treatment). All abstracts from the Medline search were screened to determine if they fulfilled the inclusion criteria by MH, aided by CV. Two authors blindly assessed a random subset of abstracts to corroborate inclusion and exclusion (JPC, AH). One hundred and sixty-one articles were chosen at random (~16 papers/year from 1998–2007 inclusive) and full texts were scrutinized in more detail.
The following were extracted and recorded from the abstracts of included articles: year of publication; first author; journal name; continent of correspondence; language of publication; disease category; study design; gene(s) studied and whether variation was in somatic (cancer) cells or in the germline and, if germline, whether related to drug absorption/distribution/metabolism/elimination(pharmacokinetic) or the drug target (pharmacodynamic) and whether the study was primarily set up to investigate the pharmacogenetic end-point. We also extracted information on: the primary outcome including whether this was the intended or an adverse effect of the drug; the number and magnitude of reported p values in each study (categorized as only non-significant p values [p>0.05], only significant p values [p≤0.05] and mixed [p values both ≤ and >0.05]); specific drugs, further classified according to the British National Formulary coding (http://www.bnf.org accessed 2009 November 10, archived URL http://www.webcitation.org/5lBYIOLVR) and the 2006 impact factor of the publication (derived from Journal Citation Reports ® ISI Web of KnowledgeSM (http://www.isiwebofknowledge.com accessed 2009 November 10, archived URL http://www.webcitation.org/5lBTT863z) grouped into 0 to 4.99, 5–9.99 and ≥10. From the 161 full-text articles, data were also extracted on: (i) genes and alleles investigated, including the mean allele frequency (MAF); (ii) outcomes, classified according to their clinical end-point into binary and continuous; (iii) the number of analyses and p values reported from gene-drug interactions.
Definition of Disease Category
Disease categories were organ-specific with the exception of (i) cancer, which encompassed any body site in which there was neoplasia, and (ii) anti-coagulation, classified as ‘cardiovascular’. The cardiovascular disease category also included acute myocardial infarction and peripheral vascular disease; neurology/psychiatry included stroke, psychosis, and depression; endocrine disease included diabetes and hyperlipidaemia (where the outcome assessed was a change in lipid level and not the effect on cardiovascular end-points).
Gene Nomenclature and Classification
Genes were named according to HUGO (HUman Genome Organisation) Gene Nomenclature Committee (HGNC, Wellcome Trust; http://www.genenames.org accessed 2009 November 10, archived URL http://www.webcitation.org/5lBCXvH6E). The classification of genes into dynamic or kinetic was checked with the Pharmacogenomics Knowledge database (PharmGKB; http://www.pharmgkb.org accessed 2009 November 10, archived URL http://www.webcitation.org/5lBChBcLk). Where it was not possible to precisely classify the specific gene according to HUGO nomenclature, an asterisk was placed after the initial characters (e.g. HTR* denotes serotonin receptor genes, of which HTR1B and HTR2A are specific examples).
A study in which the outcome investigated was the desired effect of the drug (e.g. pH lowering from use of a proton pump inhibitor) was defined as ‘intended effect’; one in which the outcome was adverse was classified as an ‘adverse effect’ (this encompassed both hypersensitivity and dose-dependent adverse reactions).
For the 161 full-text papers, outcomes were classified as binary or continuous: examples of binary were death, disease recurrence, or an episode of bleeding; examples of continuous were changes in the plasma levels of a drug, gastric pH or international normalised ratio (INR, e.g. for the monitoring of warfarin anticoagulation).
Continent of Correspondence
The continent of correspondence was determined from the Medline citation and used as a surrogate marker for the geographic location of the study.
The study design was categorized as: (i) prospective (including randomized clinical trials), (ii) case-control, (iii) cross-sectional, or (iv) meta-analysis.
Primary/Secondary Pharmacogenetic Study
A primary pharmacogenetic study was defined as one in which the title of the study or the stated aims or purpose within the text of the abstract indicated that the primary intention of the study was to investigate the effect of genetic variation on drug response. If not explicitly stated, the study was classified as a secondary pharmacogenetic study.
We excluded the following as ‘drug’ treatments: ionizing radiation, surgical procedures, non-drug-eluting stents, bone marrow transplantation, tobacco, alcohol, environmental agents or pollutants (e.g. lead), herbal remedies, dietary or lifestyle interventions including acupuncture, massage, counseling, or exercise.
U.S. Food and Drug Administration (FDA) Guidelines
We analysed the evidence-base behind the FDA list of approved pharmacogenetic tests (pre-July 2009) . The articles cited in support of FDA labeling as ‘test required’ or ‘test recommended’ were reviewed (Document S1). Tests (gene and drug pairs) were cross-referenced with the generated database. FDA recommendations were contrasted with guidelines from authoritative medical bodies.
Statistical analyses were performed using SPSS for Windows version 17.0 and Stata 10. A value of p≤0.001 was taken as significant. Frequency distributions were analysed for normality by 2-tailed Chi-Square. Impact factors were ranked by Mann-Whitney U. Sample sizes were converted into logarithmic (loge) values and means compared with unpaired student's t.
A sensitive, non-specific search strategy in Medline (see Methods) yielded 102,264 articles (Figure 1) with an additional 1,641 articles identified from other sources. 97,339 (94%) articles were annotated as reviews, editorials or letters rather than primary research, and were excluded. Of the 6,548 remaining articles, a total of 1,668 (1.6% of studies from the initial search) reported original research that fulfilled all our inclusion criteria. A much less sensitive search strategy utilising the MeSH term “pharmacogenetics” retrieved only 4674 articles, of which 183 (4%) were indexed as original research (Figure 2).
Figure 1. Flow chart of methodology for identifying pharmacogenetic studies in the systematic review.
From PRISMA 2009 guidelines .doi:10.1371/journal.pone.0007960.g001
Figure 2. Growth in publications in the field of pharmacogenetics from 1967–2007 (inclusive).
Our detailed search strategy incorporating both Medical Subject Headings (MeSH) and free-text terms (filtered for Humans and excluding Reviews/Editorials) identified 6,548 original articles (purple bars) of which 1,668 fulfilled the inclusion criteria (green bars). By contrast the total number of articles obtained based on a search using the MeSH term “pharmacogenetics” (including reviews and editorials) was 4,674, of which only 183 were original articles (red bars), indicating a ratio of approximately 1:25 of original research to commentary/review.doi:10.1371/journal.pone.0007960.g002
Characteristics of Pharmacogenetic Studies
We noted a marked increase in the number of primary pharmacogenetic research studies (and other types of article) since 1990 (Figure 2). The majority of articles reporting original research investigated variation in the germline (1327, 79.6%, Table 1) and of these, the greater proportion studied genetic variation in drug targets (pharmacodynamic studies; 804, 60.6%) rather than genes encoding proteins involved in drug handling and elimination (pharmacokinetic). Most pharmacogenetic studies were prospective in design (1496, 89.7%) with about one-half (852; 51.1%) set in Europe or Australasia and one-third in North America (511 studies; 30.7%). The most frequently investigated disease areas were cancer (456 studies; 27.3%), neurology/psychiatry (321 studies; 19.2%) and cardiovascular disease (287 studies; 17.2%) with a relative paucity of studies in infectious disease (106 studies, 6.4%) and respiratory medicine (49 studies, 2.9%). Most studies evaluated the intended effects of the drug under investigation (1190 studies; 71.6%); only one-eighth of studies (210, 12.6%) examined adverse drug effects, with pharmacokinetic rather than pharmacodynamic studies being more likely to do so (p = 2.02×10−14).
Table 1. Characteristics of pharmacogenetic studies included in the systematic review.doi:10.1371/journal.pone.0007960.t001
Genes Investigated and Number of Participants
The breadth of work and the foci of activity are illustrated by the total number of genes in each category and those most frequently studied (Figure 3). There were in total 541 genes studied (176 somatic, 305 pharmacodynamic and 70 pharmacokinetic with some overlap for 10 genes). Seven genes included studies involving over 10,000 participants in aggregate: two somatic (TP53 and non-specified karyotype mutations), 2 pharmacokinetic (MTHFR and CYP2C9) and 3 pharmacodynamic genes (ACE, AGT and APOE). About one-third (37.7%) of study participants were distributed among the 10 most frequently studied somatic genes; with the equivalent numbers in kinetic and dynamic studies being 68.5% and 41.8%, respectively. Thirteen of 70 (18.6%) kinetic genes, 22 of 305 (7.2%) dynamic genes and 12 of 176 (6.8%) somatic genes included more than 10 studies.
Figure 3. The 50 most frequently studied genes and the aggregate number of participants per gene.
(a) pharmacodynamic genes (n = 305); (b) pharmacokinetic genes (n = 70); and (c) somatic genes (n = 176). * refers to >1 gene and/or non-HUGO nomenclature.doi:10.1371/journal.pone.0007960.g003
Most Frequently Studied Gene-Drug Combinations
The 10 most studied cancer cell gene variants were TP53 and cisplatin/5-fluorouracil/paclitaxel response, ERBB2 (HER2/neu) and anthracyclines/trastuzumab response, EGFR and gefitinib response, and RAS, FLT3, ABCB1, BCL2 and t(9;22) and other karyotype and cytogenetic mutations and response to a variety of combination chemotherapy regimens. The most studied germline pharmacokinetic and pharmacodynamic genes (Figure 4) were ACE and cardiovascular drug response (n = 79), CYP2D6 and response to antidepressant therapy (n = 74), CYP2C19 and response to gastrointestinal drugs (mostly proton pump inhibitors, n = 52), MTHFR and response to nutritional drugs (predominantly folate, n = 41), ADRB2 and response to respiratory medications (n = 34), CYP2C9 and cardiovascular drugs (mainly warfarin, n = 33), APOE and response to drugs targeting the cardiovascular (n = 29) and central nervous system (CNS, n = 31), TPMT and response to chemotherapy/immunosuppression (mostly azathioprine, n = 29), and HTR* (n = 27) and DRD2 (n = 27) and response to CNS drugs. However with the exception of ERBB2(HER2/neu)/trastuzumab therapy, CYP2C9/warfarin and TPMT/azathioprine none of these genes are mandated or recommended by the FDA for pharmacogenetic testing .
Figure 4. Categories of drugs evaluated in pharmacogenetic studies of the 10 most frequently studied genes.
(a) pharmacodynamic; and (b) pharmacokinetic. Numbers represent total studies per gene and drug category, with cell color shading to emphasize value (heat matrix). CNS = central nervous system; ENT = ears, nose and throat. Drugs are classified as in British National Formulary (http://www.bnf.org).doi:10.1371/journal.pone.0007960.g004
We next focused on indices of clinical relevance and study quality. As in clinical trials, continuous outcome measures in pharmacogenetic studies are more likely to be surrogates for more clinically relevant binary outcomes. For example, the international normalized ratio (INR), an index of the anticoagulant effect of warfarin, might be used as a surrogate for the risk of a major hemorrhage, a serious adverse clinical event arising from warfarin treatment. From the representative subset of 161 full-text articles, continuous outcomes were more frequently reported than binary outcomes. Of a total of 546 reported outcomes, less than one-third (167, 30.6%) were binary, and these were more likely to be reported in studies of genetic variation in cancer cells (median binary outcomes/paper: 2, IQR 1–3.25) than germ-line studies (median binary outcomes/paper: 0, IQR 0-1).
Sample size in genetic studies can serve as an index of the quality and reliability because unless effect sizes are large, small studies may be inadequately powered to detect plausible genetic effects reliably , , . Common alleles (those with a minor allele frequency, MAF, >0.05) tend to exert smaller effects on disease risk than rare alleles , with effect sizes for binary outcomes in gene-disease association studies being odds ratios for disease risk in the range of 1.28–1.65 . Moreover, where a positive effect is seen in a small study of common alleles, a false positive association may be as or more likely than a true positive , . In the representative subset of full text articles of pharmacogenetic research, the median MAF of the variants studied was 0.12 (IQR 0.08–0.67), suggesting that similar effect sizes for binary outcomes might be expected in pharmacogenetic studies; reliable detection of effect sizes in this range would require sample sizes in the region of 3,500 . However, the vast majority of pharmacogenetic studies were far smaller (median sample size 93) and the distribution highly skewed (IQR 40–222). Moreover, there was little evidence for an increase in sample size over time (Figure 5). Although pharmacodynamic studies (median sample size 102, IQR 51–273) tended to be larger than pharmacokinetic studies (median sample size 70, IQR 25-136, p = 7.61×10−15) in neither case was the size of studies comparable to recent candidate gene or genome wide disease association studies . Larger studies tended to achieve publication in higher than intermediate or lower impact journals (p = 2.99×10−7) and articles from North America, Europe & Australasia had larger sample sizes than those from Asia (p = 2.21×10−6). However, most articles were published in journals of modest impact factor (median 4.77, IQR 2.83–8.07; 54.1% were published in journals of impact factor <5), with no clear trend for an emergence of a larger proportion of high impact factor articles over time (p = 0.861). Impact factors were higher in studies of genetic variation in cancer cells (p = 2.07×10−14) and articles from North America (p = 2.17×10−13) compared to others in their respective groups.
Figure 5. Sample size of pharmacogenetic studies from 1987 to 2007 (inclusive).
Horizontal bars designate the median, boxes indicate 25th and 75th centiles of the distribution and vertical bars represent the non-outlier range.doi:10.1371/journal.pone.0007960.g005
Reporting of Statistical Significance
Significance chasing bias, evidenced by a disproportionate reporting of extreme p values in small studies, previously affected candidate gene disease association studies . To assess whether this might also be the case in the pharmacogenetic literature, we evaluated the distribution of reported p values in abstracts of primary research articles. About one half of study abstracts (816, 48.9%) reported a p value. Three quarters of these articles reported only significant p values (608 abstracts, 74.5%). There was no difference (p = 0.926) in the size of studies among the three p value categories: median sample size (IQR) of articles reporting only non-significant p values was 99 (57–292); mixed (significant and non-significant) p values was 103 (48–252); and only significant p values was 106 (49–252). These findings were corroborated in the detailed analysis of 161 full papers (p = 0.608).
The predominance of significant p values suggests either that the prior odds of success in pharmacogenetics is higher than in most other fields of biomedical research, or that the published literature is affected by chance findings and/or publication bias . Another index of significance chasing bias is the total number of hypotheses tested by any study. One hundred and twenty five of 161 full-text articles reported a p value (Figure 6) with a median of 6 p values per article (IQR 3–12). These 161 articles had a theoretical median of 12 total reportable comparisons per study (IQR 4–29, Figure 7, calculated by number of alleles x number of drugs x number of outcomes recorded), suggesting that the potential for post-hoc subgroup analysis is large in pharmacogenetic research.
Figure 6. Distribution of p values in 161 full-text primary research articles in pharmacogenetics.doi:10.1371/journal.pone.0007960.g006
Figure 7. The theoretical number of total comparisons in 161 full-text articles.
Calculated by multiplying the number of gene alleles studied by the number of drugs investigated by the number of outcomes recorded.doi:10.1371/journal.pone.0007960.g007
Use of Meta-Analysis
Meta-analysis has been used to strengthen conclusions regarding genetic effects on disease outcomes , , . Thirty one meta-analyses of pharmacogenetic studies were identified spanning 29 genes (Table 2), 23 of which included 4 or more studies. However, a further 107 genes that were the subject of ≥4 studies had never been the subject of a meta-analysis. The majority of meta-analyses investigated variants in the germline (n = 19) with over half (n = 21) investigating intended effects and less than one-quarter (n = 7) adverse outcomes. For those genes exposed to meta-analysis, the median number of studies per gene was 22 (IQR 5–52). Six of the 7 meta-analyses in the somatic gene category (85.7%) involved the 10 most frequently studied genes, and 5 of 7 (71.4%) in the pharmacokinetic category. However, only 4 of 15 (26.7%) meta-analyses in the pharmacodynamic category involved the 10 most studied genes.
Table 2. Summary of meta-analyses of pharmacogenetic studies.doi:10.1371/journal.pone.0007960.t002
FDA-Supported Pharmacogenetic Tests
We next assessed the evidence-base for pharmacogenetic tests listed by the FDA. At the time this study was performed (pre-July 09), the FDA had published guidelines on “valid genomic biomarkers”,  classifying pharmacogenetic tests into (i) required, (ii) recommended, and, (iii) information only. In July 2009, the website was updated  with removal of the classification system, however the list of “valid genomic biomarkers” and supporting references remained largely unchanged. We based our analysis on the original guidelines with accompanying classification system (Document S1).
Of the 136 references listed by the FDA in support of pharmacogenetic testing, one article was indexed in Medline as a meta-analysis (Figure 8), 63 (46%) were annotated either as clinical trials/government-supported research or comparative studies, with the remainder (48 studies, 35%) being reviews, case reports or historical articles and 24 being unclassified. Only a small proportion of the 1668 articles identified from our search mapped to relevant FDA endorsed pharmacogenetic tests (n = 101, Table S1). FDA recommended or mandated pharmacogenetic tests were more likely to investigate adverse effects, involve pharmacokinetic genes and relate to cardiovascular disease (p = 1.43×10−16, 1.45×10−7 and 5.06×10−7 respectively).
A distinctive feature of the field of pharmacogenetics is the predominance of publications indexed as reviews, commentaries, letters and other opinion based pieces over primary research articles, whichever search strategy we used to identify articles. This may have contributed to a high expectation of the delivery of personalized medicines , ,  with modest realisation of this goal thus far. Though expanding in general, pharmacogenetic research currently centres mainly in cancer, cardiovascular and neurological/psychiatric disease with most studies being set in Europe and North America, presumably mainly among subjects of European ancestry. The relative dearth of research in other therapeutic areas (e.g. communicable disease) and among individuals of non-European ancestry, among whom there is a considerable global disease burden, may be creating an imbalance that will require addressing in future work. Even if the relevant genetic variants and effect sizes are homogeneous across different ancestral groups , differences in allele frequency can vary greatly  and such variation means that the population impact of genetic variants influencing drug response will often differ by ethnicity even if effect sizes are similar.
The major goal of pharmacogenetic research is development of genotype-based predictive tests of efficacy or toxicity. However, a prerequisite is the reliable identification of the relevant genetic loci. In genetic work, where many hundreds of thousands of hypotheses can be tested, research designs are needed that optimise the detection of true positive (while limiting the potential for false positive) association , , . Despite some high quality studies, in broad terms, there are several features of the field as a whole that suggest that only a proportion of the positive associations reported are genuine. These include: the small size of most studies coupled with the more frequent evaluation of common rather than rare variants (whose effect sizes would be predicted to be small and which therefore requires large sample sizes for their reliable detection); use of surrogate (usually continuous) outcome measures rather than more clinically relevant binary outcomes; and subgroup analyses with multiple hypothesis testing. Our study may have been limited by analysing only the abstracts of articles satisfying inclusion criteria. However, detailed data (information unlikely to be reported in abstracts) on outcome measures (binary/continuous), gene variants and reported p values were derived from the full text of a subset of 10%, which accurately reflected the span of studies in the database.
Similar problems to those we highlight were recognised in the field of genetics of common disease a decade or so ago. What followed were efforts to systematically and comprehensively collate evidence from genetic association studies, large collaborative meta-analyses, larger primary studies, more comprehensive capture of genetic variation at any given locus, independent replication, and, most recently, whole genome association studies . These developments have contributed to the discovery of many secure genetic associations that are providing new insights into disease pathogenesis, potential therapeutic targets and the possibility of developing predictive tests for disease. Several important and laudable efforts to collate and curate information on the genetic basis of drug response already exist, including those of the Pharmacogenetics Research Network . However, the challenge in identifying primary pharmacogenetic studies is illustrated by our two alternative search strategies. Our comprehensive Medline search was sensitive (yielding >100,000 articles) but non-specific, with a large number of evaluated articles not satisfying our definition of a pharmacogenetic study. However, using a specific search strategy (via the MeSH tool) the majority of articles were missed. We know of no previous attempts to systematically identify all published pharmacogenetic studies in this way but our current analysis suggests that future attempts to do so should adopt an explicit, systematic and comprehensive search strategy such as the one we have used here. The terms “pharmacogenomic” and “pharmacogenetic” have both been used somewhat interchangeably in the literature. For example the Pharmacogenomics Knowledge database (PharmGKB; http://www.pharmgkb.org/resources/forGeneralUsers/pharmacogenetics_pharmacogenomics_and_personalized_medicine.jsp accessed 2009 November 10, archived URL http://www.webcitation.org/5lBBtDJPf) defines pharmacogenetics as “the study of … varying responses to drugs and the determination of the genetic mutations underlying these variations” and pharmacogenomics as “the study of drug response in the context of the entire genome”. However, the Human Genome Project information portal (http://www.ornl.gov/sci/techresources/Human_Genome/medicine/pharma.shtml#whatis accessed 2009 November 10, archived URL http://www.webcitation.org/5lBCB8i5T) defines pharmacogenomics as “the study of how an individual's genetic inheritance affects the body's response to drugs”. These indistinct classifications are exemplified by the U.S. National Library of Medicine's ‘controlled vocabulary’ for indexing articles via MeSH terminology: “pharmacogenomics” is not a MeSH term, on entering it in Medline, all articles indexed with the MeSH term “pharmacogenetics” are displayed.
Other developments that may be helpful include: a greater use of meta-analysis, particularly where four or more independent studies of the same gene have been conducted, perhaps with an online, continuously updated database similar to those established for Alzheimer's Disease, Parkinson's Disease and Schizophrenia , , , . Other improvements might include: primary studies with larger sample sizes; wider use of haplotype tagging single nucleotide polymorphisms (SNPs); studies of rare and structural genetic variants whose effects are predicted to be larger, and which may therefore be more suited for use as predictive tests; and a greater focus on genes influencing drug handling and adverse effects, to fill gaps in knowledge .
Important studies with some of these features have been reported since the deadline we set for our literature search. For example, the identification of a SNP in the SLCO1B1 gene, encoding the organic anion-transporting polypeptide OATP1B1, as a susceptibility factor for statin-induced myopathy involved a genome-wide association analysis of 85 individuals with definite or incipient statin myopathy (and 90 controls) from a trial involving over 12,000 subjects . Here, the small size of the genome-wide association study belies the large-scale effort to identify the few subjects who suffer extreme adverse effects. This study provides a paradigm for the identification of genetic loci underlying rare but serious adverse effects of a commonly used drug. Other examples which could be studied in a similar way include heparin-induced thrombocytopaenia (frequency 0.5–2%), oesteonecrosis of the jaw from bisphosphonate treatment (prevalence 4–7% in those receiving intravenous bisphosphonates for hypercalcaemia of malignancy), and angio-oedema from angiotensin converting enzyme inhibitors. Because of the large genetic effect sizes that might be detected with this approach (for example an odds ratio of 17 for statin myopathy in SLCO1B1 CC homozygotes), predictive tests may be more likely to emerge, though the rarity of the adverse effect means that rigorous assessment of the cost-effectiveness of the approach would first be required. Larger scale candidate gene studies, , ,  are also providing much more secure evidence on loci influencing both drug response and adverse effects that might form the basis of predictive testing for dose adjustment or avoidance of toxic treatments.
As more reliable information begins to emerge on alleles influencing drug response from larger, better designed whole genome and candidate gene studies, focus will need to shift to the critical evaluation of the predictive performance of genetic tests in clinical practice, including studies of cost-effectiveness. These evaluations will require use of different metrics to those conventionally reported in discovery-based genetic studies (such as odds ratios or proportion of variance explained) , , . Instead, sensitivity and specificity, predictive values and the generation of multivariate models that include genotype will need evaluating , . In some cases, the most robust evaluation of the effectiveness of genetic tests may need to come from randomised trials comparing health outcomes among people randomised to pharmacogenetic testing or no testing, together with cost-effectiveness analyses as are now common when evaluating the usefulness of interventions. In concert, these efforts should help realise the promise of personalised medicines with resultant improvements in healthcare. Our recommendations for pharmacogenetic research are summarised below.
Recommendations for Future Research in Pharmacogenetics
Primary research in pharmacogenetics should:
- give due emphasis both to adverse as well as intended effects of drugs
- be appropriately powered
- examine clinically-relevant end-points
- be conducted among individuals of non-European as well as European ancestry
- include studies of currently neglected drugs and disease areas
- enhance the likelihood of identification of large effect sizes necessary for the generation of usefully predictive tests through the study of rare or structural genetic variants, and/or more extreme phenotypic differences in response or toxicity
- ensure comprehensive SNP typing where candidate loci are studied
- utilise whole genome analysis where mechanisms are uncertain
- avoid post-hoc subgroup analysis, except where justified and powered, and report the findings with due caution
- include evidence of independent replication
- exploit existing large randomised controlled trial datasets as a resource for pharmacogenetic evaluation (e.g. SLCO1B1 variants and statin-induced myopathy, based on the SEARCH trial involving 12,064 participants) 
Mechanisms should exist for:
- encouraging reporting null findings from high-quality studies
- systematically and comprehensively collating, archiving and disseminating reports of pharmacogenetic research, to highlight continuing gaps in knowledge and promote successes
- encouraging high quality updated systematic reviews and meta-analyses of pharmacogenetic research
Promising genotype-based predictive tests emerging from primary research should be:
- re-evaluated in independent prospective studies
- assessed against clinically relevant outcomes
- evaluated using the appropriate metrics for diagnostic, screening and predictive tests
- tested where appropriate in randomised trials
U.S. Food and Drug Administration (FDA) Table of Valid Genomic Biomarkers in the Context of Approved Drug Labels (website pre-July 2009)
(3.01 MB PDF)
U.S. Food and Drug Administration (FDA) mandated or recommended pharmacogenetic tests pre-July 2009
(0.06 MB DOC)
Conceived and designed the experiments: MVH AH JPC. Performed the experiments: MVH TS CV AH JPC. Analyzed the data: MVH AH JPC. Contributed reagents/materials/analysis tools: MVH TS LS AH JPC. Wrote the paper: MVH LS AH JPC.
- 1. Connor S (2003) Glaxo chief: Our drugs do not work on most patients. The Independent. Available: http://www.independent.co.uk/news/science/glaxo-chief-our-drugs-do-not-work-on-most-patients-575942.html Accessed 2009 November 10, Archived URL: http://www.webcitation.org/5lBCqy0gg.
- 2. Pirmohamed M, James S, Meakin S, Green C, Scott AK, et al. (2004) Adverse drug reactions as cause of admission to hospital: prospective analysis of 18 820 patients. BMJ 329: 15–19.
- 3. Hunter DJ, Altshuler D, Rader DJ (2008) From Darwin's finches to canaries in the coal mine–mining the genome for new biology. N Engl J Med 358: 2760–2763.
- 4. Lemonick MD, Cray D, Park A, Thomas CB, Thompson D (2001) Brave New Pharmacy. Time. Available: http://www.time.com/time/magazine/article/0,9171,998963-1,00.html Accessed 2009 November 10, Archived URL: http://www.webcitation.org/5lBD8FmTx.
- 5. Marr K (2008) A Glimpse Into Personalized Medicine of the Future. The Washington Post. Available: http://www.washingtonpost.com/wp-dyn/content/article/2008/09/28/AR2008092802482.html Accessed 2009 November 10, Archived URL: http://www.webcitation.org/5lBDFAeE7.
- 6. Pollack A (November 8, 2005) A Special Drug Just for You, At the End of a Long Pipeline. The New York Times. Available: http://www.nytimes.com/2005/11/08/health/08phar.html Accessed 2009 November 10, Archived URL: http://www.webcitation.org/5lBVt5O2X.
- 7. Roses AD (2000) Pharmacogenetics and the practice of medicine. Nature 405: 857–865.
- 8. Goldstein DB (2009) Common genetic variation and human traits. N Engl J Med 360: 1696–1698.
- 9. Goldstein DB, Tate SK, Sisodiya SM (2003) Pharmacogenetics goes genomic. Nat Rev Genet 4: 937–947.
- 10. (2006) Table of Valid Genomic Biomarkers in the Context of Approved Drug Labels. FDA (Created 2006 September 15, Updated 2008 September 10, Removed 2009 June). Accessed 2009 January 12, Archived URL: http://www.webcitation.org/5l6cpblur (older version). See Document S1 (recent version prior to removal of website).
- 11. McClain MR, Palomaki GE, Piper M, Haddow JE (2008) A rapid-ACCE review of CYP2C9 and VKORC1 alleles testing to inform warfarin dosing in adults at elevated risk for thrombotic events to avoid serious bleeding. Genet Med 10: 89–98.
- 12. Hynicka LM, Cahoon WD Jr, Bukaveckas BL (2008) Genetic testing for warfarin therapy initiation. Ann Pharmacother 42: 1298–1303.
- 13. Shurin SB, Nabel EG (2008) Pharmacogenomics–ready for prime time? N Engl J Med 358: 1061–1063.
- 14. Ioannidis JP, Ntzani EE, Trikalinos TA, Contopoulos-Ioannidis DG (2001) Replication validity of genetic association studies. Nat Genet 29: 306–309.
- 15. Ioannidis JP, Trikalinos TA, Ntzani EE, Contopoulos-Ioannidis DG (2003) Genetic associations in large versus small studies: an empirical assessment. Lancet 361: 567–571.
- 16. Colhoun HM, McKeigue PM, Davey Smith G (2003) Problems of reporting genetic associations with complex outcomes. Lancet 361: 865–872.
- 17. Ioannidis JP, Gwinn M, Little J, Higgins JP, Bernstein JL, et al. (2006) A road map for efficient and reliable human genome epidemiology. Nat Genet 38: 3–5.
- 18. Higgins JP, Little J, Ioannidis JP, Bray MS, Manolio TA, et al. (2007) Turning the pump handle: evolving methods for integrating the evidence on gene-disease association. Am J Epidemiol 166: 863–866.
- 19. Ioannidis JP, Boffetta P, Little J, O'Brien TR, Uitterlinden AG, et al. (2008) Assessment of cumulative evidence on genetic associations: interim guidelines. Int J Epidemiol 37: 120–132.
- 20. Pennisi E (2007) Breakthrough of the year. Human genetic variation. Science 318: 1842–1843.
- 21. Moher D, Liberati A, Tetzlaff J, Altman DG (2009) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 6: e1000097.
- 22. Ioannidis JP (2005) Why most published research findings are false. PLoS Med 2: e124.
- 23. Ioannidis JP (2008) Effect of formal statistical significance on the credibility of observational associations. Am J Epidemiol 168: 374–383; discussion 384–390.
- 24. Ioannidis JP, Trikalinos TA, Khoury MJ (2006) Implications of small effect sizes of individual genetic variants on the design and interpretation of genetic association studies of complex diseases. Am J Epidemiol 164: 609–614.
- 25. Bodmer W, Bonilla C (2008) Common and rare variants in multifactorial susceptibility to common diseases. Nat Genet 40: 695–701.
- 26. Hindorff LA JH, Mehta JP, Manolio TA (2009) A catalog of published genome-wide association studies. National Human Genome Research Institute. Available: www.genome.gov/26525384 Accessed 2009 November 10, Archived URL: http://www.webcitation.org/5lBRp4wFx.
- 27. Khoury MJ, Little J, Gwinn M, Ioannidis JP (2007) On the synthesis and interpretation of consistent but weak gene-disease associations in the era of genome-wide association studies. Int J Epidemiol 36: 439–445.
- 28. Altshuler D, Hirschhorn JN, Klannemark M, Lindgren CM, Vohl MC, et al. (2000) The common PPARgamma Pro12Ala polymorphism is associated with decreased risk of type 2 diabetes. Nat Genet 26: 76–80.
- 29. Frank B, Wiestler M, Kropp S, Hemminki K, Spurdle AB, et al. (2008) Association of a common AKAP9 variant with breast cancer risk: a collaborative analysis. J Natl Cancer Inst 100: 437–442.
- 30. Sagoo GS, Tatt I, Salanti G, Butterworth AS, Sarwar N, et al. (2008) Seven lipoprotein lipase gene polymorphisms, lipid fractions, and coronary disease: a HuGE association review and meta-analysis. Am J Epidemiol 168: 1233–1246.
- 31. (2009) Table of Valid Genomic Biomarkers in the Context of Approved Drug Labels (Updated 2009 July 8 and 2009 August 18). FDA. Available: http://www.fda.gov/Drugs/ScienceResearch/ResearchAreas/Pharmacogenetics/ucm083378.htm Accessed 2009 November 7, Archived URL: http://www.webcitation.org/5l6d2Q4LH.
- 32. Ioannidis JP, Ntzani EE, Trikalinos TA (2004) ‘Racial’ differences in genetic effects for complex diseases. Nat Genet 36: 1312–1318.
- 33. Limdi NA, Arnett DK, Goldstein JA, Beasley TM, McGwin G, et al. (2008) Influence of CYP2C9 and VKORC1 on warfarin dose, anticoagulation attainment and maintenance among European-Americans and African-Americans. Pharmacogenomics 9: 511–526.
- 34. Pharmacogenetics Research Network. National Institute of General Medical Sciences, NIH. Available: http://www.nigms.nih.gov/Initiatives/PGRN Accessed 2009 November 10, Archived URL: http://www.webcitation.org/5lBSkNtMF.
- 35. SchizophreniaGene (SZGene), Schizophrenia Research Forum. Available: http://www.schizophreniaforum.org/res/sczgene Accessed 2009 November 10, Archived URL: http://www.webcitation.org/5lBSs9qHy.
- 36. Bertram L, McQueen MB, Mullin K, Blacker D, Tanzi RE (2007) Systematic meta-analyses of Alzheimer disease genetic association studies: the AlzGene database. Nat Genet 39: 17–23.
- 37. Frodsham AJ, Higgins JP (2007) Online genetic databases informing human genome epidemiology. BMC Med Res Methodol 7: 31.
- 38. Tang S, Zhang Z, Kavitha G, Tan EK, Ng SK (2009) MDPD: an integrated genetic information resource for Parkinson's disease. Nucleic Acids Res 37: D858–862.
- 39. Woodcock J, Lesko LJ (2009) Pharmacogenetics–tailoring treatment for the outliers. N Engl J Med 360: 811–813.
- 40. Link E, Parish S, Armitage J, Bowman L, Heath S, et al. (2008) SLCO1B1 variants and statin-induced myopathy–a genomewide study. N Engl J Med 359: 789–799.
- 41. Colombo S, Rauch A, Rotger M, Fellay J, Martinez R, et al. (2008) The HCP5 single-nucleotide polymorphism: a simple screening tool for prediction of hypersensitivity reaction to abacavir. J Infect Dis 198: 864–867.
- 42. Klein TE, Altman RB, Eriksson N, Gage BF, Kimmel SE, et al. (2009) Estimation of the warfarin dose with clinical and pharmacogenetic data. N Engl J Med 360: 753–764.
- 43. Mega JL, Close SL, Wiviott SD, Shen L, Hockett RD, et al. (2009) Cytochrome p-450 polymorphisms and response to clopidogrel. N Engl J Med 360: 354–362.
- 44. Simon T, Verstuyft C, Mary-Krause M, Quteineh L, Drouet E, et al. (2009) Genetic determinants of response to clopidogrel and cardiovascular events. N Engl J Med 360: 363–375.
- 45. Jakobsdottir J, Gorin MB, Conley YP, Ferrell RE, Weeks DE (2009) Interpretation of genetic association studies: markers with replicated highly significant odds ratios may be poor classifiers. PLoS Genet 5: e1000337.
- 46. Janssens AC, Aulchenko YS, Elefante S, Borsboom GJ, Steyerberg EW, et al. (2006) Predictive testing for complex diseases using multiple genes: fact or fiction? Genet Med 8: 395–400.
- 47. Kraft P, Wacholder S, Cornelis MC, Hu FB, Hayes RB, et al. (2009) Beyond odds ratios - communicating disease risk based on genetic profiles. Nat Rev Genet.
- 48. Bromley CM, Close S, Cohen N, Favis R, Fijal B, et al. (2009) Designing pharmacogenetic projects in industry: practical design perspectives from the Industry Pharmacogenomics Working Group. Pharmacogenomics J 9: 14–22.