More on the Seroquel BOLDERs

Dawdy over at Furious Seasons has corresponded with Dr. Michael Thase (see the entire post regarding the correspondence here), the lead author on the BOLDER II study, which I discussed earlier. My main complaint was that the study generated its effect sizes (the statistical measure of how large of an effect was generated by treatment) in an unconventional manner. They used a method called mixed model repeated measures analysis (MMRM). Let’s not get into the statistical details too deeply, suffice to say that this is not the conventional method for generating effect sizes.

One of Thase’s responses to questions regarding the effect size calculation was that MMRM is becoming “rapidly adopted as the method of choice for clinical trial data sets” – however, I have not seen it used to the extent that I’d call it the “method of choice.”

The main issue remains: the apparent magnitude of the effect of Seroquel on depressive symptoms increased by 50% when MMRM was used. That makes the treatment look more effective than when the more conventional method is used. Thase responded that “we did not report effect sizes using the [traditional] LOCF method largely because we didn't anticipate that someone would actually want to see them” – why wouldn’t people want to see them – given that the MMRM method inflated the apparent effect of the treatment by 50%, this is certainly news worth reporting in the article!

Mind you, the interpretation of effect sizes has always been based on traditional methods – we have no idea if the interpretation of effect sizes based on MMRM is accurate. I can buy into using MMRM as a method of testing statistical significance (was the treatment more effective than a placebo) but not into using MMRM to generate effect sizes (how much more effective was the treatment than a placebo). When a change of 50% can be seen in the magnitude of treatment depending on the type of analysis used, it is controversial and requires further explanation.

Thase has done a lot of good work and this post should not be taken as a personal attack on him. Indeed, I was glad to see him respond to Dawdy -- that is a good sign of collegiality and openness and he is commended for responding. I strongly disagree with the way data were presented and I don’t buy Thase’s argument that the traditional analysis was just not interesting enough to publish.