Introduction

The recognition that health services, like other activities, have limited resources has lead to the widespread adoption of cost-effectiveness criteria, a collection of methods intended to help decision-makers make the best choices for the adoption of or disinvestment in health technologies under conditions of uncertainty, conflicting objectives and resource constraints [1]. Healthcare decisions and policies are being increasingly driven by cost-effectiveness analyses (CEA), and in several countries the incremental cost-effectiveness ratio (ICER) has become a key determinant of whether a health technology will be publicly funded [2]. Thus, a large number of economic evaluation studies, many of them conducted or sponsored by the manufacturers of the technologies assessed, has been published in recent years [3–7].

Although there is a substantial body of literature for both randomized trials and observational studies indicating the existence of a positive and statistically significant relationship between sources of funding and study results [8–12], this so-called “sponsorship bias” has received relatively little attention in other scientific areas such as cost-effectiveness research (despite the exploiting of opportunities to make diverse assumptions [13]). When available, most studies analyzing a sponsorship bias in cost-effectiveness literature deal with high-cost drugs (mainly oncologic drugs) or new technologies with little clinical evidence at a population-level. However, for certain drugs with a well established use in clinical practice, sponsorship bias has hardly been addressed, including for example the case of specific cardiovascular therapies. Statins (or HMG-CoA reductase inhibitors) lower cholesterol levels, reduce cardiovascular (CV) events and mortality, and are generally well tolerated, with serious side effects being unusual [14,15]. While evidence of the possible benefits of statins in low-risk populations accumulates [16] and prices decline because of the expiry of most patents and the rapid introduction of low-price generics [17], many countries are witnessing a significant expansion in statin prescribing. However, controversies about the real-world clinical effectiveness of statin therapy in primary CV prevention and also regarding the proportion of people eligible for treatment (e.g. financial constraints and the narrow benefit-risk balance in people at low CV risk) still persist [18–27]. In this context, this study aims to examine the association between the funding source of published cost-effectiveness analyses of statin use for both primary and secondary CV prevention and the study conclusions.

Methods

Results

Our initial literature search yielded 462 citations. After screening titles and abstracts, 145 full text reports were obtained and 9 additional articles were identified from the reference lists of retrieved articles and hand searching. Overall, 75 articles met our inclusion criteria (see Figure 1). The full references for the articles included are listed in the Additional Files (Table S1: “List of Included References”). The PRISMA checklist is provided in the Additional Files (Table S2: “PRISMA Checklist”).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Flow diagram of study selection

process.

https://doi.org/10.1371/journal.pone.0069462.g001

Table 1 details the characteristics of the articles reviewed. The studies, most of which were based in Europe and North America, were principally published in journals with a high-impact factor (Tables 1 and 2). Fifty-two articles (69.3%) compared statins versus non-active alternatives. The most frequent statin involved in the studies (defined as the intervention) was atorvastatin (n=21, 28.0%), followed by pravastatin (n=18, 24.0%) and simvastatin (n=16, 21.3%) (Table 3). Studies were funded primarily by industry (64.0%), followed by non-profit or government organizations (18.7%). Secondary CV prevention represented 42.7% of the articles, followed by primary CV prevention (38.7%) or both primary and secondary CV prevention (18.7%). From the 75 articles included, in 44.0% of them the first author was employed in academia and 41.3% of studies had at least 1 author with a pharmaceutical company or consulting firm affiliation. The summary characteristics and results of selected articles are provided in the Additional Files (Table S3: “Summary Characteristics of Selected Articles” and Table S4: “Base case ICERs for Selected Articles”).

Overall, industry-sponsored studies were much less likely to report unfavourable or neutral conclusions than non-sponsored ones (0% [0 out of 54] versus 37.1% [13 out of 35], p<0.001) (Table 4). For primary CV prevention, the proportion of studies with unfavourable or neutral conclusions was 0% [0 out of 24] for industry-sponsored studies versus 57.9% [11 out of 19] for non-sponsored studies (p<0.001). No statistically significant differences were identified for studies evaluating secondary CV prevention (0% [0 out of 30] versus 12.5% [2 out of 16], respectively; p=0.222). The journal impact factor was not significantly associated with study conclusions, as shown in Table 5.

A total of 1,612 ICERs were obtained from the 75 CEAs included in our study. Figure 2 shows the variability in the estimated incremental costs per QALYs/LYGs gained by funding source and prevention category. For primary CV prevention, ICERs reported in industry-sponsored studies were more likely to fall below the $50,000 willingness-to-pay threshold than in non-sponsored ones (61.6% [253 out of 411] versus 22.3% [135 out of 605], p<0.001). For secondary CV prevention, the majority of ICERs considered fell below the threshold in both industry and non-industry sponsored studies (91.2% [386 out of 423] versus 68.8% [119 out of 173], respectively) (p<0.001).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Variations in cost-effectiveness results by funding source and prevention category: a) Primary CV prevention and b) Secondary CV prevention.

CV: Cardiovascular.

Note: Each dot represents an incremental cost (in US$) per QALY/LYG in the reviewed articles. The horizontal line represents the willingness-to-pay threshold.

https://doi.org/10.1371/journal.pone.0069462.g002

Discussion

This systematic analysis identified a significant association between the conclusions of statin cost-effectiveness and industry sponsorship. Pharmaceutical company-sponsored studies were significantly less likely to reach neutral or unfavourable conclusions than non-pharmaceutical company-sponsored studies. In fact, all pharmaceutical company-sponsored studies concluded that the corresponding statin had a favourable cost-effectiveness ratio compared to any of the other agents (including competing statins) or placebo. An alternative exploratory analysis showed that ICERs reported in industry-sponsored studies were more likely to fall below the $50,000 willingness-to-pay threshold than those reported in non-sponsored ones. The difference in the “positive values” was especially striking in primary CV prevention.

These results are consistent with other studies assessing the relationship between the source of funding and the cost-effectiveness findings, specifically for oncologic drugs [29–33], neuropsychiatric drugs [34–36], venous thromboembolism pharmacologic prevention [37], drug-eluting stents [38], bisphosphonates for osteoporosis treatment [39], Pap tests for cervical cancer diagnosis [40]. They are also consistent with the CEAs submitted by manufacturers to health technology assessment agencies [5,6], and/or reviews selecting general cost-effectiveness studies [4,7,41,42]. Regarding the sponsorship bias of cost-effectiveness of statin use in CV prevention, our results confirm and extend those reported by Franco and colleagues [43]. They conducted a review including 24 studies between 1990 and 2002 with 216 ICER, showing that differences in cost-effectiveness ratios between industry and non-industry sponsored studies were higher the lower the level of CV risk.

The mechanisms through which authors (and their sponsors) may potentially obtain more favourable cost-effectiveness results have been little studied, but several possibilities have been suggested [30,34,44–49]. For example, the so-called “publication bias” against untraceable or unpublished papers with non-significant or unfavourable results could be present in some CEA research [6]. Another source of sponsorship bias could be that dragged from efficacy studies, which was shown by Bero and colleagues [8], who reported in a cross-sectional study of 192 randomized trials that comparative trials of statins were more likely to report outcomes favouring the sponsor’s statin compared to the comparator drug.

Regarding journal characteristics, it could be argued that they could be associated with the reliability of study findings and conclusions as the quality of the reporting could vary among journals. In the study conducted by Bell and colleagues [4], a CEA review of the English-language articles indexed in PubMed/MEDLINE since 2001, the authors observed that the more quoted journals with an impact factor above 4 were less likely to publish favourable results (e.g. ratios below $20,000-50,000/QALY) than less quoted journals with a lower impact factor. In our research, however, the journal impact factor was not significantly associated with study conclusions. In fact, these differences with our results are probably part of the high prevalence of studies published in high-impact journals in our sample (e.g. at least three-quarters of our selected articles were published in top journals positioned in the higher quartiles of their specialty), this not being a particularity in the cardiovascular area.

If sponsorship bias is a relevant explanation for our results, making decisions (e.g. selection of certain drugs in national formularies) based on results from industry-sponsored evaluations might be considered (arguably wrongly) to be a futile exercise. In such hypothetical situation, an economic evaluation could make healthcare services even more inefficient, increasing spending on medication prescription without obtaining clear benefits in the health of the population.

Our study has several limitations. First, the extent of the relationships between the project, the authors and the sponsors was limited to the information publicly available in the corresponding paper. There were no further inquiries or attempts to verify the truthfulness of the conflict of interest statements. And only information about funding and authors’ affiliation was taken into account in the main analyses, without considering other possible sources of conflict of interest. Additionally, although similar methods have been used in earlier studies [7–9,32,33,35], we acknowledge there could be some amount of subjectivity involved in assessing whether the conclusions of a paper were biased towards the sponsor’s drug. To minimize this potential reporting bias we used explicit inclusion criteria, pre-defined study definitions, and data coding and consensus among researchers in the case of disagreement. Second, study eligibility was restricted to the English and Spanish languages for practical reasons. However, Egger and colleagues [50] pointed out the existence of a language bias in the publication of randomized trials from non-English countries, with a greater likelihood of publication of significant results in English language journals. It is difficult to determine to what extent this bias affects cost-effectiveness research in non-English journals, which is usually focused on a local market. In any case, the language bias is a limitation to consider in this study. Third, while applying the same economic models to different populations or countries is a common practice in CEAs and, therefore, some degree of autocorrelation between studies is expected, we analyzed each study as an independent observation. This effect may have distorted the strength of the association between the type of funding and cost-effectiveness conclusions. We emphasise that caution is needed when comparing the incremental cost-effectiveness figures, as we were unable to analyse further by stratifying them based on baseline cardiac risks.

Generally, publication planning by the pharmaceutical industry has been suggested to undermine the research process in several ways, as marketing interests may influence the research process at multiple different times: during decisions over which studies to conduct, study design, analytic approach, writing of manuscripts, and whether or not to publish research findings [51,52]. It is possible that in the case of high cost drugs, industry funded economic evaluations are designed from the outset to suggest greater cost-effective. These market access strategies are relevant because they could generate confusion about the true therapeutic added-value of new drugs, and result in a considerable loss of credibility of the cost-effectiveness literature, finally leading to concerns about the applicability of economic analyses and impeding the use of economic evaluations in medical decision-making.

Unfortunately, it does not seem to be a problem that can be easily solved and it is hard to identify any effective strategy to limit the potential for manipulation. Full disclosure of the authors’ conflict of interest is the most widespread approach and some journals apply restrictive policies on industry-sponsored cost-effectiveness studies [53,54]. The use of guidelines or good practices for economic evaluation is strongly recommended as a possible solution by journals, experts and professional societies [55], and several editorialists have also proposed strengthening the peer-review processes and making them more transparent [43,44,54]. All of these strategies are probably sound and indispensable in the current context, but they all also have important weaknesses. Policies on conflict of interest so far have not been sufficient to reduce sponsors’ control of the results of cost-effectiveness studies, and the extensive collaborative networks between manufacturers, contracted consulting firms, academia and healthcare centres [56] create situations that allow industry sponsorship to be concealed. The use of methodological guidelines does not prevent slipping "reasonable favourable assumptions" into economic models (as Evans stated “researchers know what sponsors want to hear and what will affect the probability of subsequent support” [13]). Additionally, the accurate peer review of economic models can be extremely laborious [57]. In fact, to redo with local data some parameters from a previous model developed for a pharmaceutical company and to post a “new” local or updated article can be much less costly than the evaluation of that article by the journal’s reviewers [58]. Public reporting of models and unhindered access to the methods used to produce cost-effectiveness estimates (so that they can be examined in detail) could help. If public authorities really want CEAs to become a credible tool for supporting their own decision-making, they will have to encourage and finance the conducting of independent cost-effectiveness research.

In summary, our study demonstrates an important sponsorship bias in the cost-effectiveness of statins for CV prevention, and adds a new piece to the growing body of literature suggesting a major distortion in our knowledge about the true efficiency (“good value for money”) of drugs and medical technologies. The evaluation of medical technologies is not only, or even primarily, the evaluation of their cost-effectiveness. The dimensions of safety, comparative effectiveness and other socially pertinent criteria should be reconciled in determining health priorities. But economic evaluation, by its intrinsic logic for identifying and assessing all costs and benefits, is one of the most informative approaches to making decisions for funding (or not) health technologies, especially when working with imperfect information [59]. It is in this ability to operate with imperfect information that many of its advantages and the potential for interested manipulation reside. Correcting or mitigating these risks is probably one of the most important issues in the cost-effectiveness of medical technologies in order to ensure they reach their full potential to support decision-making processes.

Supporting Information