Shadow Statisticians

An interesting new study is available in PLoS Medicine (Gotzsche et al., 2007). Due to the open access of all PLoS articles, you can read it in full for free and I encourage all of you to do so.

In this study, the authors examined 44 trials supported by industry. They examined whether there were contributors listed on the original study protocol (submitted prior to the start of a study) who did not appear on the final published manuscript, which was of course published after the completion of the trial. They found evidence for ghost authorship in 75% of cases. In almost all ghost-suspected cases, the protocol listed that a company statistician would help interpret the data, yet the statistician in question was not listed as an author on the publication.

Why does this matter? This is important because when you are reading an article, you are expecting that the listed authors were responsible for the study design, data collection, data analysis, interpretation, and writing of the paper. Look at the following example:

A paper is listed as having “Independent” Academic Author X, Company Statistician Y, and “Independent” Academic Author Z. It concludes that a drug is highly effective and very safe. Given that it has a company statistician, do you trust its results entirely? Well, if the company statistician is removed, leaving only two “independent” academic authors, then the potential influence of the statistician is removed from plain sight. We are thus more apt to assume the analyses were done objectively, not with an eye toward cherry picking data to show positive results.

To quote from the PLoS article, “[Ghostwriting] might happen because the study ‘looks’ more credible if the true authors (for example, company employees or freelance medical writers) are not revealed.”

But wait, there’s more: The 75% figure is almost certainly an underestimate. Why? Because this study could not detect the presence of ghostwriters – it is rare for a company to mention in a study protocol that a ghostwriter will be used to write a draft of the study manuscript for publication. So how often are ghostwriters used? Healy estimated that among psychiatry papers, perhaps 50% are ghostwritten. If we look at papers that were written up by professional medical writers and/or used company statisticians yet failed to acknowledge them properly in the final manuscript, we are clearly looking at an exceedingly high percentage of papers!

Conclusion: To be clear, a company statistician may conduct analyses properly and a medical writer may write a manuscript that accurately and fairly represents the study’s data as well as the research design’s strengths and limitations. Yet there have been many cases where the statistics in industry-supported studies have been dodgy (for example, here and here) and in which a medical writer has written a draft that is clearly overly optimistic (such as this and this) regarding a treatment.

Analyzing data in a “friendly” manner to a product is just another way in which industry-supported science has largely merged with marketing. I don’t believe that hiding the identities of people who analyzed the data and of the hired gun(s) who wrote the manuscript can be viewed in any other way.