Defending the Hiding of Negative Clinical Trial Data

In the wake of the New England Journal of Medicine study that revealed a sizable discrepancy between the raw data on antidepressants and the data published in medical journals, a few people have jumped to the defense of drug companies. In this post, I will examine their arguments and compare them with evidence. As an extra feature, the DrugWonks' disconcerting love of Vioxx will be discussed toward the end of the post.

"Nathan", who posts frequently on the excellent Pharmalot blog, made several comments. Here's one of them:

For those of you that haven’t written up a scientific article for publication, I’ll make you aware of a few things:
1) Journals don’t accept an article just because it is written. Journal editors scrutinize it and submit it to reviewers who further scrutinize it — very frequently rejecting articles that aren’t conclusive or are poorly written.
2) Scientists write scientific articles — not public relations representatives.
3) Scientists generally don’t waste weeks (or months) of their time writing up articles that don’t prove anything and probably won’t be accepted by the journal. A negative result is generally meaningless in science. Do you think Thomas Edison wrote up articles about the 900 filaments that failed to light a light bulb?

Now I’ll ask again: Is it any surprise that the negative clinical trials were not written up as publications?

Here’s one example: The Paxil trial may have tried doses of 10, 25, and 50 mg. The 10 and 25 mg dose failed to show an effect. The 50 mg dose did show an effect. Why should anyone waste their time writing up a journal article explaining why the 10 and 25 mg dose failed? It’s obvious that drug levels just weren’t high enough to observe an effect.

The problems with the above post:

1. Well, let's face it, peer review does not exactly catch everything. In fact, peer review often results in the publication of very poor studies, like the infamous Paxil Study 329. But it actually appears that Nathan's point is that studies that "aren't conclusive or are poorly written" don't get published. So is he saying that the positive studies of SSRIs were more likely to written better than studies that found negative results? Not sure that makes much sense. Perhaps he is implying that a negative study is not "conclusive" -- well, neither is a positive study, for that matter. When one considers the small benefit of antidepressants over placebo, then even positive studies have not exactly yielded conclusive evidence of anything particularly impressive.

2. Nathan is either naive on this point or is being dishonest. It is well known that ghostwriters are frequently used in the publication process (1, 2, 3, 4, 5). This is not a problem when they are just helping a poor writer piece together a readable manuscript, but this is often a huge problem when one considers that ghostwriters often have a huge conflict of interest in that they are hired by drug companies to pen papers that will paint their product in a favorable light.

3. "A negative result is meaningless in science." That is an amazing comment. Truly amazing. So if 10 trials are conducted on a drug, and one turns up positive, then I suppose everyone should prescribe that drug because the nine negative results were "meaningless." Yeah, that makes a lot of sense.

3.5. The point about dosing would make sense except that it is irrelevant to the case at hand. Remember, Nathan is commenting specifically on the antidepressant studies that did not get published, and he is hypothesizing that the authors of that study included doses that were not approved as effective. Perhaps he missed page 253 of the article, which states:

We included data pertaining only to doses later approved as safe and effective; data pertaining to unapproved dosages were excluded.

Oh, so the authors did not include data for unapproved doses. So much for that critique. Reading a study before offering critiques of it: A good idea.

Nathan in another comment:

I understand how this APPEARS to be a smoking gun. But I’ll state again why I believe it is not: Let’s say I do a clinical trial with Paxil at 10 mg (once per day), 20 mg (once per day), 20 mg (twice per day), and 50 mg (twice per day). That’s a total of 4 trials. Only the 50 mg dose works. Are the other three trials counted as “negative” evidence that the drug doesn’t work? Of course not.

See above for why this critique holds no water.

Nathan:

As Bob pointed out, editors like to accept articles that make their journal look good. NEJM is probably not going to accept a bunch of articles about clinical trials that didn’t work. That doesn’t mean the editors are in some sort of under-the-table deal with the company. It simply means that they are looking out for the best interest of the JOURNAL, not the public.

I think we’re all missing the point. Of course scientists like to publish things that work, not things that fail. The point is that ALL clinical trial data (the good, the bad, and the ugly) is available to the public, the FDA, and anyone else interested. Whether or not the data makes it into a journal is irrelevant.

All clinical trial data is available to the public? Cue the laugh track. Turn it up a little. No, a little more still. Ah, there we go. Who is he kidding? There is absolutely no such thing as a repository for clinical trial data. That is part of the reason why the problem with ghosted science exists in the first place. There are clinical trial registries, but go ahead and check out clinicaltrials.gov. One can outline a study on such a registry then report the results in whatever manner one deems fit, or not at all.

Enter the DrugWonks: And, as you would have guessed, the dependable, fair and balanced DrugWonks weighed in as well. Here are pieces of Robert Goldberg's post:

The NEJM of medicine recycles the old story that many of negative studies about antidepressants were not published. That doesn't affect whether the drugs work or not. It does add to the distortion of what a negative study is and why they are negative. Most of the time they are negative because they simply confirm the hypothesis. Other times they are poorly designed or small studies of little statistical power. They don't prove that the drugs fail. There is a difference. Taken together they can often help guide who responds to what medicines or why not...which again is why we need the Critical Path. To suggest that the failure to publish negative studies is part of a coverup is wrong and leads to fearmongering once again. We have been down this road. And journalists are once again raising unfounded fears about the safety and efficacy of drugs...leading people to die because they stop taking medicines because of the fearmongering the media has engaged in regarding vaccines, SSRIs, Avandia, Vioxx and Vytorin

As for the argument that studies were negative simply because their sample sizes were small, please read the following from the article (page 254):

The difference between the sample sizes for the published studies (median, 153 patients) and the unpublished studies (median, 146 patients) was neither large nor significant.

In other words, Goldberg's analysis is wrong. As I stated above: Reading a study before offering critiques of it: A good idea. As for "Most of the time they are negative because they simply confirm the hypothesis." -- What the hell does that mean? Confirm what hypothesis? This is the sort of seemingly random statement that I have run across frequently on the DrugWonks site. As for the negative studies being poorly designed (so the positive studies are thus designed better?), this seems wrong as well. These studies aren't particularly complex. One group gets drug, the other gets placebo. Participants are assigned to groups randomly. In theory, participants are unaware of if they are receiving drug or placebo. It appears that there were not large differences in how these studies were designed. In any case, Goldberg provides not one single shred of evidence to support his claims. Nathan, mentioned above, also provided no evidence to support his critiques of the NEJM study.

Bring on the Vioxx Love. Goldberg then goes to a new level. He wrote, in part:

And journalists are once again raising unfounded fears about the safety and efficacy of drugs...leading people to die because they stop taking medicines because of the fearmongering the media has engaged in regarding vaccines, SSRIs, Avandia, Vioxx and Vytorin

Let's just focus on one of these medicines: Vioxx. I should mention that as I type this, I am quaking with rage. Goldberg is either incredibly ignorant or incredibly dishonest. For a person who claims expertise in matters of drugs, he should know much better than this. He is claiming that people died because they stopped taking Vioxx. He is apparently serious. Lord Jesus, tell me that this man is joking... First of all, how many people have died because they switched from one minor painkiller to another? Bob, please provide the data. PLEASE.

But (and this is where the rage is coming from), Bob, since you are a Drug Wonk, which would imply at the very least a firm understanding of important and highly publicized findings from the medical literature, how many people died because they did take Vioxx? According to a study by David Graham and colleagues published in the Lancet:

Using the relative risks from the abovementioned randomised clinical trials and the background rates seen in NSAID risk studies, an estimated 88000–140000 excess cases of serious coronary heart disease probably occurred in the USA over the market-life of rofecoxib. [If even a third of these people died, well, you do the math...]

But taking this drug off the market resulted in people dying? Are you *!@! kidding me? Because Graham is a "fearmonger," one of the favorite epithets thrown around on the DrugWonks site, how about we not take his word for it... What have other researchers found? One meta-analysis found an increased risk for heart attacks with COX-2 inhibitors relative to placebo and to some other drugs. Another meta-analysis also found elevated heart attack risk for Vioxx. How about this meta-analysis, which found increased risk for both cardiovascular and renal events on Vioxx? Another meta-analysis found cardiovascular events were related to Vioxx use.

So, Bob, there's my evidence on Vioxx. Where's yours? It's clear that Bob and I differ on a number of points. I think reasonable people can disagree on many items, but Vioxx is not one of them. A drug that has been clearly and repeatedly linked to serious health problems -- and Bob is defending it, while calling those who criticize it "fearmongers." Nice game plan.

A More Reasonable Critique. The Last Psychiatrist offers a more reasonable analysis, suggesting that academic authors did not want to publish the negative studies because publishing negative studies would make them look incompetent -- the mainstream culture of medicine would ask who runs a trial and finds negative results? And even if the studies were submitted to a journal, then they were likely to be rejected by peer reviewers and editors.

This may be true to an extent, but I tend to believe that some negative studies would have been published. There have always been journal editors who have some ability to think critically and publish material that runs counter to that of mainstream medicine. Sure, some of the studies would have been rejected a time or two, but I think they would have been published at some point. While I think his analysis is intelligent (and it thankfully does not involve the term "fearmonger"), I think it only offers a partial explanation.

Missing the Boat. I could certainly have missed something, but I have not even seen a lame attempt to critique part of the study that is quite damning. Remember, the study found that drug companies changed their primary outcome measures and statistical analyses in between submitting to the FDA and submitting for journal publication. This resulted in inflated effect sizes for every antidepressant. Kind of a big deal, as the medical literature ends up suggesting the drugs are more beneficial than they actually are. Not even DrugWonks has mentioned this rather major point.

Oh, and one more thing for Bob Goldberg: Before engaging in a variety of tactics that border on slander regarding Roy Poses of Health Care Renewal (in a recent often nonsensical post that I won't honor by linking to it), you might want to improve your own credibility.

If you missed it, read my post about the NEJM study that some folks are now critiquing.

Guilt By Association

Over at Drug Wonks is a recent post by Robert Goldberg that would be funny if it weren't so insulting. I have previously discussed how Drug Wonks appears to be a part of the drug industry PR machine. The latest is at least the second swipe that the Drug Wonks have taken at Carlat. It would appear that Carlat is guilty of guilt by association according to Goldberg's reasoning. Carlat was previously guilty of the heinous sin of criticizing the drug industry.

Carlat has linked to the AHRP in his list of sites labeled "Great Pharma Information". This induced a rant from Goldberg in which he called the folks at AHRP "whack jobs." I have likewise linked the AHRP on my blogroll, so I suppose I am equally guilty. Goldberg is also hot under the collar about Carlat linking to Prescription Access Litigation. Those greedy lawyers linked to PAL are just suing drugmakers to line their own pockets, or at least that's the Goldberg argument. The problem is that it has taken litigation to unearth the infamous Zyprexa documents, which have suggested strongly that Lilly's marketing of Zyprexa was waaaaaaay out of line (1, 2, 3 ). Are the lawyers set to make out like bandits in these lawsuits? Sure. But how else are corporations to be held accountable for pushing drugs for inappropriate uses or masking side effects of their products?

And more from Goldberg...

Let Carlat ally himself with anti-science extremists and tort lawyers. In the wake of the SSRI-suicide debacle I will put my trust in the so-called pharma-academic complex any day

Oh, Goldberg is apparently in the SSRI's are never linked to suicidality camp? Not sure, but it looks that way. He may want to refresh his knowledge on such matters here or here. Oh, but I guess that actually looking at the data from clinical trials and noting that it indicates SSRIs are related to a greater incidence of suicidal acts than placebo makes me a "whack job." That's for y'all to decide, I suppose.

Advertising as Education: Carlat Hits Home Run

Daniel Carlat is the chief of the Carlat Report, a newsletter which provides CME credits for physicians and is (gasp) sometimes critical of the drug industry and its marketing practices.

Carlat has an op-ed in the New York Times where he presents several examples of how drug companies, in the name of education, have misled physicians. I'd like to summarize what he wrote, but he really said it best. Please check it out. It covers a lot of important material in relatively few words. He also has a simple solution -- any CME paid for by drug companies should not count for educational credit for physicians. I couldn't agree more!

As y'all know, I am highly critical of current CME practices, which allow physicians to maintain their medical licenses based upon their soaking up sugarcoated drug company advertising which masquerades as education. For examples, please read an earlier post on a pro-Geodon piece and a puff piece pushing transcranial magnetic stimulation for depression. I am very pleased that the Times ran Carlat's piece.

John Mack noted that the Drug Wonks blog (predictably) came to the rescue of Big Pharma by calling Carlat's piece "hysterical." Of course, Robert Goldberg (the Drug Wonks author in question) provided no evidence -- he also said

Oh, and if I were Carlat, I would drop the characterization "Unbiased" from the description of his expensive newsletter. He is as biased as anyone. Anyone who calls his colleagues money launderers is biased.

Oh gee, we're all biased, so who knows what's right? It's all so relativistic. Please. I think that many pharma bloggers (including myself), have marshaled sufficient evidence to indicate that Big Pharma has often placed marketing well ahead of patient benefit. For documentation, um, start reading a bunch of posts on this and similar sites. Yes, we're all biased to one extent or another -- that's why we should stick to facts rather than label industry critics as "hysterical" or worse. And the facts indicate that CME needs reform. Call me hysterical if you want.

Hat Tip: Furious Seasons

Welcome to the PR Machine

The conflict of interest Tilt-A-Whirl continues at breakneck speed. Yet it's even more than a conflict of interest. After reading a few great posts by fellow bloggers recently, I've become somewhat discouraged at how dissenting voices are rarely heard amidst a buzz of industry-funded cheerleading.

What am I talking about?

Drug Wonks: The blog Drug Wonks frequently defends drug companies and their practices, while heaping derision upon those who challenge the drug industry. I'm not saying that Dr. Steve Nissen, of recent Avandia meta-analysis fame, is perfect, but MAN, the scorn they are heaping upon him is over the top. Here are a couple comments about Nissen made on their site:

1. "Dr. Nissen was alternately defensive, evasive and hesitant" in Congressional hearings.
   
2. "Diedtra Henderson interviews Steve Nissen who responds -- sort of-- to my blog calling him 'small and craven' as he shifts positions and engages in self promotion in a campaign to become defacto FDA commissioner"
3. Drug Wonks blogger Robert Goldberg has also stated that Nissen's true interest is in becoming FDA commissioner, and that this current Avandia is essentially a publicity stunt to help him move up the ranks of FDA commissioner contenders.

Who are the Drug Wonks? To steal, er, borrow from Health Care Renewal's fine post on this topic of PR,

The President of the Center for Medicine in the Public Interest, where Goldberg works, and which runs the blog called DrugWonks, is Peter J Pitts, whose day job is to be Senior Vice President for Global Health Affairs at the big public relations firm Manning, Selvage and Lee. Manning, Selvage and Lee has many big pharmaceutical accounts.

Okay then. So drug companies hire a firm to help them manage their reputations. The president of the PR firm runs a blog on the side that just, by sheer coincidence, happens to cast dispersions on those who criticize the drug industry.

There is still more to the story. Robert Goldberg, who blogs regularly at Drug Wonks, was a fellow at the Manhattan Institute, and he is now Vice President of the Center for Medicine in the Public Interest. What is the Manhattan Institute? It is a thinktank that advocates for free-market reforms. It would be fair to say that the Manhattan Institute favors a more limited role for the FDA. So who funds this Manhattan Institute? They're not telling. But it would appear that they receive the lion's share of their funding from a variety of corporations and foundations that advocate for laissez-faire capitalism.

So it's like this... The Manhattan Institute advocates for the "free market" to save health care. But behind the scenes, the Manhattan Institute exists only because a variety of interests, from highly rich individuals to large drug companies, need to push their message. The Institute is created to give an outlet for their message. I don't know who funds the Center for Medicine in the Public Interest. I'm betting it's not the Tooth Fairy.

The Point: Because they have money, corporate interests can buy the time of people like Peter Pitts and Robert Goldberg, who can be counted on to reliably talk in a friendly manner about big corporations. I'm not saying that Goldberg, Pitts, and company advocate for reforms that happen to support the interests of their sponsors just because they are paid to do so. Far from it. My guess is that these folks had their free-marketeering views long before they signed on with any sort of thinktank. They likely honestly believe what they are writing, but it also can't hurt that they are being paid by these corporate interests -- most people won't offend their sponsors, as there can be untoward consequences for doing so (1, 2).

The Golden Rule of Speech: He/She Who Has the Most Gold Has His/Her Message Heard Most Frequently.

People from industry friendly thinktanks are frequently featured on TV newscasts, in newspaper stories, on on op-ed pieces. They are cited as "experts" -- why are they considered experts? Well, gee, the person came from an "Institute" -- he/she MUST be well-informed!

Those of us in the healthcare media world who are more skeptical of the behavior of drug companies are typically not nearly as well-funded. In fact, if you look around the healthcare blogosphere, you'll find that most of us ain't getting a nickel for our work (Rost, Dawdy, Jack Friday, myself, Roy Poses, Aubrey Blumsohn, and many more). Yes, Rost wrote a book, but nobody was paying him to say anything. Anyway, does our working for peanuts make us masochists? Maybe, but that's a topic for another day. It's great that Rost was featured in Fortune (as was PharmaGossip), but such publicity is rare. While the industry-funded folks are regularly featured in higher-profile media (1, 2, 3), the rest of us are left writing for a smaller (but growing) audience. Sure, folks at Public Citizen get press regularly, but as a general rule, the frequently-cited "experts" are those with some connection to industry or industry-related public relations. Is this going to change?

Additional Reading: If you really want your socks to be knocked off regarding the huge role of public relations firms in American society, you really should read Toxic Sludge is Good for You.

I'll be taking a "safe, gentle psychotropic" while I ponder how our small blogger voices can somehow reach maximum amplification.

Don't Diss Berenson

Drug Wonks has a post saying that Alex Berenson's latest reporting on the Zyprexa scandal is likely erroneous because he is pulling information from just a few Lilly documents. Indeed, it is stated...

Once again, Berenson uses the "one paper out of several thousands of documents" approach to depict Lilly as a criminal corporation.

Further, it is added

On the heels of intense criticism of the NYT coverage of the Duke rape case and the paper's falling earnings, the Berenson reporting is another example of the Times decline.

OK. Here's my challenge. I'd like Drug Wonks or, well, anyone to read the following posts. Read the Lilly documents that are freely available and are linked within said posts. Then talk about how Lilly is a good corporate citizen, and did not promote Zyprexa off-label. Here you go...

• Demented Marketing of Zyprexa?
• Marketing Zyprexa as the New Mood Stabilizer...
• Zyprexa Off-Label Promotion (?)
• Zyprexa Off-Label Marketing (?): Part 2

As for the "well, we haven't seen every single document" argument, please see my earlier post for why I think that argument is weak.