The Cost of Medical Evidence

There are a number of mechanisms to turn the medical literature into useful evidence, such as meta-analysis, systematic review and comparative effectiveness studies (see our previous posts on this topic). And it is well known that these efforts take significant time, and therefore cost.

But how much do they cost, really? As data scientists, we realize this is an empirical question. So without further ado, here our (simple) model for how much is spent, per company per year, for the major pharmaceutical companies to turn the literature into evidence (useful or not).

To be very clear: throughout this article we will use the word “study” to interchangeably mean a systematic review, a comparative effectiveness study or a meta-analysis.

How much does one “study” cost? [1]

To begin our model, we needed to quantify two major factors in generating one of these “studies” (e.g., comparative effectiveness, meta-analysis or systematic review):

  1. How long does it take?
  2. How many people are involved?

If we can get these two figures, along with some estimated salary information, we can quantify how much it costs to produce a single study. Fortunately for us, Borah, et al. (2017) published just such a result. According to their paper, it takes, on average, 16 months and 5 co-authors to produce one review.      Of course, producing studies doesn’t take 100% of a scientist’s time. In fact, UCSF’s guide states[2] that reviews take a minimum of 6 months, working 10-20 hours per week. So, let’s take an average across those publications, yielding 11 months to produce the review (6 months from UCSF, 16 from Borah, et al.), and assume that researchers spend 15 out of their 40 hours per week on it (the mid point between 10 and 20 hours). This means they spend 37.5% of those 11 months on the studies, which translates into working 4.125 months per year. Put another way, 5 co-authors working 4.125 months per year is the same as 1.72 scientists working for an entire year.

Therefore, at an average salary of $80,530 per scientist,[3] each study costs the equivalent of $138,511.60.

How many do we do?

We now have a sense of how much each study costs – but how many are actually done? If we can estimate that number, we can get a sense of how much it costs, per year, to do all the studies we might need.

There are actually two factors to consider. First, we obviously want to estimate how many studies a company publishes. Second, not all studies undertaken actually produce useful results, so we also want to estimate how many studies were done, but not published. This will estimate the number of times a company did the work, but didn’t publish the final result.

To answer our first question, we took a list of the top 10 pharmaceutical companies by revenue[4] and searched for systematic reviews, meta-analyses or comparative effectiveness studies on PubMed, where at least one of the authors was affiliated with that company.[5] The table below shows the queries and results, noting that we only search for studies published within the last 5 years (through 2017).

Company Studies last 5 years PubMed Query (date range: 1/1/2013 – 12/31/2017)
Roche 421 (“Meta-Analysis”[pt] OR “systematic review” OR “comparative effectiveness”) AND (Roche[Affiliation])
J&J 90 (“meta-analysis”[pt] or “systematic review” or “comparative effectiveness”) and (“Johnson and Johnson”[affiliation] OR “Johnson & Johnson”[affiliation])
Pfizer 638 (“meta-analysis”[pt] or “systematic review” or “comparative effectiveness”) and (Pfizer[affiliation])
Novartis 575 (“meta-analysis”[pt] or “systematic review” or “comparative effectiveness”) and (Novartis[affiliation])
Sanofi 263 (“meta-analysis”[pt] or “systematic review” or “comparative effectiveness”) and (Sanofi[affiliation])
GSK 364 (“meta-analysis”[pt] or “systematic review” or “comparative effectiveness”) and (GlaxoSmithKline[affiliation])
Merck 392 (“meta-analysis”[pt] or “systematic review” or “comparative effectiveness”) and (Merck[affiliation])
AbbVie 146 (“meta-analysis”[pt] or “systematic review” or “comparative effectiveness”) and (AbbVie[affiliation])
Bayer 232 (“meta-analysis”[pt] or “systematic review” or “comparative effectiveness”) and (Bayer[affiliation])
Abbott 203 (“meta-analysis”[pt] or “systematic review” or “comparative effectiveness”) and (Abbott[affiliation])
Average 332.4  

So, on average, these companies wrote 332.4 papers over the last five years, which is an average of 66 studies per year!

Again, not all studies end up being published, so the number above doesn’t reflect all of the studies undertaken. An interesting paper by Shuit and Ioannidis (2016) looked into this issue and found that (for networked meta-analyses) 44% of the time a meta-analysis wasn’t published.[6] So, if we assume our 10 companies are publishing 66 studies on average, then they aren’t publishing 52 studies (44% of the total studies would be 66 divided by 100% – 44%), for a grand total of 118 studies undertaken per year.

So what does it all mean?

So, we estimated that the top 10 pharmaceutical companies work on 118 studies per year, at a cost of $138,511.60 per study, for a total of $16,344,298. Let’s assume we are off by 10% or so, to make the number nice and round; therefore these studies cost these large companies roughly $15,000,000 per year.

Crucially, if automation could save these companies just 33% of that effort, they would each save $5,000,000 per year!

Now, if your company is much smaller, just scale appropriately! (to reflect that you need fewer studies) So, if you are half as large, then automation can probably save you $2,500,000, and if you are 10% of the size of large pharmaceutical company, then automation can probably save you $500,000 on your efforts (the equivalent of 5 scientists!).

There is a lot of text and math in this post, so we’ve created the following info-graphic below to highlight the main points.

And, of course, if you would like to know more, please reach out to us, we would love to hear from you!

References:

Borah R, Brown AW, Capers PL, et al. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open 2017;7:e012545. doi: 10.1136/bmjopen-2016-012545

Schuit E, Ioannidis JP. Network meta-analyses performed by contracting companies and commissioned by industry. Systematic Reviews. 2016;5:198. doi:10.1186/s13643-016-0377-3.

Footnotes:

[1] Again, remember that we will use the term “study” here as a catch for any aggregated analysis such as meta-analysis, comparative effectiveness study, etc.

[2] https://guides.ucsf.edu/c.php?g=375744&p=3041343

[3] https://www.bls.gov/ooh/life-physical-and-social-science/medical-scientists.htm

[4] https://en.wikipedia.org/wiki/List_of_largest_pharmaceutical_companies_by_revenue

[5] Of course, this methodology has many flaws, including the fact that if only 1 of the authors is company affiliated, the cost is much less to that company

[6] Of the 174 analyses they looked at, 45 were published, 40 were planning for publication and 13 were going to be submitted as HTA, leaving 76 unpublished for various reasons.