Shadow Statisticians

An interesting new study is available in PLoS Medicine (Gotzsche et al., 2007). Due to the open access of all PLoS articles, you can read it in full for free and I encourage all of you to do so.

In this study, the authors examined 44 trials supported by industry. They examined whether there were contributors listed on the original study protocol (submitted prior to the start of a study) who did not appear on the final published manuscript, which was of course published after the completion of the trial. They found evidence for ghost authorship in 75% of cases. In almost all ghost-suspected cases, the protocol listed that a company statistician would help interpret the data, yet the statistician in question was not listed as an author on the publication.

Why does this matter? This is important because when you are reading an article, you are expecting that the listed authors were responsible for the study design, data collection, data analysis, interpretation, and writing of the paper. Look at the following example:

A paper is listed as having “Independent” Academic Author X, Company Statistician Y, and “Independent” Academic Author Z. It concludes that a drug is highly effective and very safe. Given that it has a company statistician, do you trust its results entirely? Well, if the company statistician is removed, leaving only two “independent” academic authors, then the potential influence of the statistician is removed from plain sight. We are thus more apt to assume the analyses were done objectively, not with an eye toward cherry picking data to show positive results.

To quote from the PLoS article, “[Ghostwriting] might happen because the study ‘looks’ more credible if the true authors (for example, company employees or freelance medical writers) are not revealed.”

But wait, there’s more: The 75% figure is almost certainly an underestimate. Why? Because this study could not detect the presence of ghostwriters – it is rare for a company to mention in a study protocol that a ghostwriter will be used to write a draft of the study manuscript for publication. So how often are ghostwriters used? Healy estimated that among psychiatry papers, perhaps 50% are ghostwritten. If we look at papers that were written up by professional medical writers and/or used company statisticians yet failed to acknowledge them properly in the final manuscript, we are clearly looking at an exceedingly high percentage of papers!

Conclusion: To be clear, a company statistician may conduct analyses properly and a medical writer may write a manuscript that accurately and fairly represents the study’s data as well as the research design’s strengths and limitations. Yet there have been many cases where the statistics in industry-supported studies have been dodgy (for example, here and here) and in which a medical writer has written a draft that is clearly overly optimistic (such as this and this) regarding a treatment.

Analyzing data in a “friendly” manner to a product is just another way in which industry-supported science has largely merged with marketing. I don’t believe that hiding the identities of people who analyzed the data and of the hired gun(s) who wrote the manuscript can be viewed in any other way.

Greedy and Ghostly Scientists

Story one: Zachary Stowe, psychiatrist at Emory University becomes Charles Nemeroff, Jr. Read all about it the Carlat Psychiatry Blog and University Diaries. And check out the WSJ Health Blog as well. The gist is that Stowe apparently did not report all of his external income from his many pharmaceutical industry gigs. Better yet, he was a frequent speaker for GlaxoSmithKline, which had the gall to cancel two of his commercial talks. He then wanted GSK to pay him even though he wasn't going to give the speeches. Read the relevant emails toward the bottom of this document. After reading about Stowe, refresh your memory about Golden Goblet Lifetime Achievement Award Winner, former Chair of Psychiatry at Emory University: Charles Nemeroff. Is there something in the water at Emory? Or is that just how we roll in modern academic psychiatry? Stowe is hereby nominated for a coveted Golden Goblet for his string of emails in which he attempted to shake down GlaxoSmithKline. Sometimes I think that the only thing worse than drug companies are the narcissistic academics who they employ as "key opinion leaders." Not all key opinion leaders are jerks; some are probably even able to reasonably balance their industry cash with being good scientists. But Stowe didn't really portray himself as Mr. Nice Guy in his string of soon to be infamous emails.

Oh, and this little gem:

"Especially disturbing is an email between employees at GSK and a public relations (PR) firm that the GSK hired. The email was titled “For your review/Paxil Breast Milk Press Release” and states:

"[P]lease review the attached press release and forward me any comments/edits.
As you may know, Dr. Stowe is on board for publicity efforts and NAME
REDACTED and I are coordinating time to meet with him next week to arm him
with the key messages for this announcement, which is slated for early February.
We are sending the release for your review at the same time in efforts to secure
distribution on Emory letterhead (as you know, would provide further credibility
to data for the media)."

In his testimony, Dr. Stowe confirmed that the press release was written by the PR
firm and concerned his research on Paxil and its presence in breast milk. He also
explained that placing the press release on Emory letterhead, as opposed to GSK letterhead, would make the data more credible to the public."

If I have this straight, Stowe was willing to place a press release written by a PR firm hired by GSK on official university letterhead to enhance its credibility. Apparently he wasn't concerned about his own credibility. Read the full document of Senator Charles Grassley's investigation of Dr. Stowe.

Part 2: Enter the Ghostwriters

One snippet, then go to Bloomberg for the rest:

Ensuring that medical journal articles presented Zyprexa study results in a positive light was one way for Lilly to reach its sales goal, company officials said in its plan, according to the documents. To do that, Lilly officials hired ghostwriters to prepare submissions to journals such as Progress in Neurology and Psychiatry, according to the unsealed documents. “The paper for the Progress in Neurology and Psychiatry supplement has been completed and sent to the journal for peer review,” Kerrie Mitchell, an employee of the public relations agency Cohn & Wolfe, wrote in a Feb. 23, 2001, e-mail to Michael Sale, a Lilly marketing official. The message was among the unsealed files. “We ‘ghost’ wrote this article and then worked with author Dr. Haddad to work up the final copy,” Mitchell said in the e- mail. Eric Litchfield, a spokesman for Cohn & Wolfe, didn’t immediately return a call requesting comment.

The Bloomberg story is based on a recently released set of internal Lilly documents. That's right -- more Zyprexa documents are on the loose. And the first round of documents provided some good stuff (1, 2, 3), so I can't wait to see what kind of chicanery will be revealed by the latest round. In one sense, it's not exactly news that Lilly ghostwrote Zyprexa papers. We all know that ghostwriting is rampant. How else do key opinion leaders get their names on dozens of papers per year when they are also flying around the country pimping drugs, holding administrative meetings, and doing all sorts of other tasks? But it's nice to have it officially documented that Lilly was playing the ghostwriting game with Zyprexa.

Key Opinion Leader Syndrome

I ran across a rather hilarious article from Medical Hypotheses, in which David Healy described "Krapelin-Fraud Syndrome", which I have also dubbed "Key Opinion Leader Syndrome." See below for the diagnostic criteria.

In line with current neo-Kraepelinian thinking, we put forward operational criteria for this new disorder for provisional inclusion in ICD-XI or DSM-V. An affected subject should meet at least 2 of criteria A–D and 2 more from criteria E–J. Fulfillment of all criteria A–D in the absence of any other features of the disorder will make the diagnosis, although this may represent a syndromal variant.
(A) A pervasive pattern of travelling to scientific conferences and talking about research data that he has had no involvement in generating.
(B) Episodic logosagnosia.
(C) Unusual abilities to compartmentalise information.
(D) Will have a significant number of ‘‘ghost-written” articles.
(E) Actively seeks admiration by peers and subordinates.
(F) An exaggerated sense of own talents, which can be inferred from expectations of recognition as an expert in the absence of commensurate achievements. Happy in the role of opinion leader.
(G) Has a sense of entitlement, i.e. unreasonable expectations of favorable treatment from symposium and congress organisers.
(H) Liable to profound dysphoria if not involved with the ‘‘academic action”.
(I) May be unreasonably envious of the scientific achievements of others and is liable to denigrate these. Would also be unhappy if his colleagues had appeared on ‘‘educational” videos and he had not.
(J) Is unaware of the disorder quality of the syndrome.

Two case studies are included, one of which reads in part:

One of the striking features of his lecturing is the dissociation between his reputation as a critical and skeptical lecturer when dealing with topics on the main programme of the meeting and the extent to which he may be prepared to offer apparently enthusiastic and uncritical endorsement for a compound in a satellite symposium. Very frequently this uncritical endorsement will involve the recycling of outdated ideas, which it is difficult to believe that either B or indeed many of his audience can conceivably believe and which indeed he may contradict within the hour at another symposium.

Hmmmm. Enthusiastic and uncritical endorsement of [insert product name here]. That reminds me of a post or two I've written... I made a rough list of symptoms for KOL Syndrome in July 2008. Different symptoms, but same idea.

Go Blackwell

Blackwell Publishing has issued a set of guidelines for its journal editors in order to increase transparency and smoke out conflicts of interest. They advise their editors to make sure that authors describe their contributions to the research project in order to lessen the frequency of ghost authorship and honorary authorship.

They also mention that duplicate publication is a no-no and insist that editors be free of influence from business managers. Consider the following situation: Both a journal editor and the journal’s business manager are aware that a clinical trial (well-designed or not) can bring in beaucoup bucks in reprints (to be distributed to docs by drug reps) whereas another study, that may be more scientifically sound, may yield zilch in reprint income. Financially, the choice is obvious, but the scientific choice may yield an opposite conclusion. I thank Richard Smith as well as Lexchin and Light for bringing the issue of journal reprints to the fore.

I sincerely hope that this is more than window dressing. Blackwell stopped short of actually inserting teeth into the guidelines by refusing to advise editors to ban bad actors. So if an author hides conflicts of interest, selectively reports data (e.g., 10 measures are used but only five are actually mentioned as being used when the study is written up), uses an unacknowledged ghostwriter, tags a big name coauthor onto a study though the coauthor does not merit authorship, and so forth, there is no real punishment. Without teeth, regulations often end up as just a way for an organization to say “Look how hard we’re trying to combat the problem” while giving a wink and a nod to the offending parties. Without either tight enforcement of punitive policies, such as banning offending authors from future publication, or a huge groundswell of ethics, I'm afraid the policy will have little effect.

Nonetheless, I commend Blackwell for their effort. It's a good step. Kudos.

Please read the whole story at Inside Higher Education. There’s much more to it than I’ve written in this post.

Corruption and Fraud in Medical Journals

Richard Smith, ex-editor of the British Medical Journal, has just published a book titled The Trouble with Medical Journals in which he blasts for their lax oversight. The following is an excerpt from PharmaGossip's nice summary.

He said: "It is increasingly apparent that many of the studies journals contain are fraudulent, and the scientific community has not responded adequately to the problem of fraud."He added: "I went away to Venice to write this book and I was rather taken aback by how negatively it turned out. When I put together all the evidence on journals I was surprised by the extent of the problems."

Dr Smith went on: "Medical journals have increasingly become creatures of the drug industry. The authors of studies in journals have often had little do with the work they are reporting."The use of ghost writers by pharmaceutical companies is rampant and many studies have conflicts of interest that are not declared."

I've not had the good fortune of reading Dr. Smith's book, but it's officially added to my to-do list!

Window Dressing at Stanford

Stanford's New Conflict of Interest Policy: No Coffee Mugs and Pens, But Consulting Contracts, Directorships, and Stock Options are Fine

I'd like to take credit for the above headline and related story, but I can't. The credit goes to Health Care Renewal for the story on Stanford's much ballyhooed PR campaign, er, new prohibitions on interactions between affiliated physicians and the drug industry.

With much fanfare, as reported by the New York Times, Stanford University announced it will "prohibit its physicians from accepting even small gifts like pens and mugs from pharmaceutical sales representatives under a new policy intended to limit industry influence on patient care and doctor education." Also prohibited will be "accepting free drug samples and from publishing articles in medical journals that are ghost-written by industry contractors. The policy would also apply to sales representatives from makers of medical devices and other companies, not just pharmaceutical companies. Company representatives would be barred from areas where patient treatment and doctor education occur...."

But...

"On the other hand, " The new policy does not cover consulting agreements between faculty members and companies aimed at developing drugs or medical devices. Those are governed by an existing conflict-of-interest policy."

Interested readers should follow the link to HCR's site for the full story.

Ghosts, Goblins, and Serotonin: Boo!

In an earlier post, I noted that I thought a key opinion leader had contradicted himself across two articles regarding the role of serotonin in depression. One reader posted a comment that challenged my assertion, to which I reply via this post. The reader, “Alan,” stated, in part, that

He only said what he said -- that [serotonin] is clearly disordered and deficient in many if not most people with depression. And he is right. There's overwhelming evidence for that. (There's also great evidence for the therapeutic value of serotonergic interventions in depression, which he did not mention.) That's not to say that other things are not playing a role, or that serotonin is the sole problem area -- the "single fundamental neurobiological defect". He only said what he said. And this blogger is jumping all over him. Why?

Okay. Two issues here. One: Did the key opinion leader (Charles Nemeroff) contradict himself? Two: Is serotonin deficient in depression? This post will deal with issue two – people can read the old post and decide for themselves if Nemeroff’s statements were contradictory or if I was in error. You decide.

Part 1: Does a Serotonin Deficiency Cause Depression?

Let’s break this thing down to make it really simple to understand. Statement 1 from Alan: [serotonin] is clearly disordered and deficient in many if not most people with depression... There's overwhelming evidence for that.

Really? Drug companies certainly use serotonin to market their antidepressants, but is there solid evidence for a serotonin imbalance in depression? Actually, no.

Despite making excellent marketing copy, studies have found no consistent abnormality in serotonin in depressed people. Doubt me? Read this excellent article by Lacasse and Leo (published in PLoS Medicine) that describes the gap between the marketing of serotonin in depression and the scientific literature.

One quote from the PLoS Medicine article:

Consider the medical textbook, Essential Psychopharmacology, which states, “So far, there is no clear and convincing evidence that monoamine deficiency accounts for depression; that is, there is no ‘real’ monoamine deficit” [44]. Like the pharmaceutical company advertisements, this explanation is very easy to understand, yet it paints a very different picture about the serotonin hypothesis.

But since SSRI’s impact depression and also impact serotonin, depression must be due to a serotonin deficiency. Um, no. Again, I’ll leave it to Lacasse & Leo:

With direct proof of serotonin deficiency in any mental disorder lacking, the claimed efficacy of SSRIs is often cited as indirect support for the serotonin hypothesis. Yet, this ex juvantibus line of reasoning (i.e., reasoning “backwards” to make assumptions about disease causation based on the response of the disease to a treatment) is logically problematic – the fact that aspirin cures headaches does not prove that headaches are due to low levels of aspirin in the brain. Serotonin researchers from the US National Institute of Mental Health Laboratory of Clinical Science clearly state, “[T]he demonstrated efficacy of selective serotonin reuptake inhibitors…cannot be used as primary evidence for serotonergic dysfunction in the pathophysiology of these disorders” [12].

I could quote the article extensively, but I’d prefer that you read it yourself. Whether you have a scientific background or not, it’s easy to understand and it shows that the serotonin emperor is wearing no clothes. Don’t take my word for it. After you’ve read the article, if you’d like to do your own independent investigation on the topic, go ahead. Please report your findings showing a strong link between serotonin dysfunction and depression right here in the comment section. I’m waiting.

It is true that variations in the serotonin transporter gene can predispose people toward experiencing depression. I don’t deny that. And given that our understanding of the brain is still rather primitive, there may be some point where we figure out that serotonin plays a certain role in depression. But at this point, there is no evidence supporting a specific serotonin deficiency in depression. Again, please correct me if you disagree.

Part 2: Do SSRI’s Work?

Another piece from Alan’s comment:

There's also great evidence for the therapeutic value of serotonergic interventions in depression.

Like what? Try that about 80% of the drug effect is replicated by placebo – there is about a 20% difference in efficacy between placebo and antidepressant (Kirsch et al., 2002). Is that “great evidence” of efficacy? It’s more encouraging than 0% better than placebo, but I remain less than fully convinced. And about those sexual side effects and increased risk of suicidal thinking and suicide attempts… If depression was really due to poor serotonin function, then one would expect treatments that increase serotonin transmission would have a much stronger advantage over placebo.

I appreciate Alan’s comment as well as its timing. It is only appropriate on Halloween, I should discuss the serotonin-depression link, as it is about as well supported scientifically as many of the ghost stories often told on such a holiday.