Efficacy: Here’s what the authors said…
Apripiprazole 10 mg/day was efficacious, and safe for psychosis associated with AD, significantly improving psychotic symptoms, agitation, and clinical global impression.
Quickie stats lesson. In determining if there is a “statistically significant difference” between two groups, a large factor is the size of the sample. Just because a difference is “statistically significant” does not imply that the difference actually means anything of value. With a large sample size (as was seen in the present study), very small effects can become “statistically significant.”
Next, let's discuss the size of these treatment effects -- how helpful was Abilify? Mean change on the Neuopsychiatric Inventory – Nursing Home Version was 17.6 points for the 10 mg/day Abilify group and 13.0 points for the placebo group. With the knowledge of the mean change and the standard deviations from both groups, one can easily calculate the effect size, which indicates the magnitude of the treatment effect (i.e., did people get a little better or a lot better?). The effect size (using Cohen’s d) was .14. The general guideline is that d = .20 is a small effect, so less than small – I guess you could call it miniscule.
How about other measures used in the study? Here are a few...
- Neuropsychiatric Inventory – Nursing Home Version Psychosis subscale: Effect size = .19
- Clinical Global Impressions – Severity: Effect size = .12
- Clinical Global Impressions – Improvement: Effect size = .10
- Brief Psychiatric Rating Scale: Effect size = .15
- Cohen-Mansfield Agitation Inventory: Effect size = .17
No matter how you slice it, the effects were all very small. Let’s also do a common sense test. The Clinical Global Impressions – Improvement Scale asks one question. The wording may vary slightly across studies, but one example is:
Rate total improvement whether or not in your judgment it is due entirely to drug treatment. Compared to his/her condition at admission to the project, how much has he/she changed?
In the current Abilify study, the average advantage for people taking Abilify 10 mg/day versus placebo was 0.3 points. There are seven points on the rating scale. For example, at one point on the rating scale is minimally improved and the next point is moderately improved. So the average patient on Abilify was 0.3 points closer to moderately improved compared to minimally improved relative to someone on a placebo. Can you say ooooh, big deal?
Anyone who has taken a basic course in statistics should be either laughing or crying (your choice) at this point. What makes this particularly egregious is that the first two authors are academic researchers who absolutely must have received this lesson dozens of times during their research training, so there is no pleading ignorance on their part.
Statistical Significance: Okay, maybe this is a smaller point. But when I plug their numbers (means, SD, N) into a t-test, I find that some of the differences they labeled as statistically significant are no longer statistically significant. Yes, they used a different statistic, which was apparently useful to them, but I thought I’d at least mention that if someone chose to use a different analysis, the data may have turned out as not statistically significant. In any case, the benefits of Abilify as shown in this study are minimal at best.
Safety: Okay, so Abilify does not work very well, but at least it won’t kill you, right? 18 people died across the course of the study, including 3% of the placebo group and 7% of the Abilify 10 mg/day group. Cerebrovascular adverse events were reported for four patients in the 10/mg Abilify group and zero people in the placebo group. As has become nearly an essential disclaimer in studies these days, the authors write: “Eighteen deaths occurred during the study, none of which were considered to be related to study medication. [my emphasis]” Of course not. The study is funded by a Bristol-Myers Squibb, and the company and its “independent” academic investigators determine if the drug could have caused any deaths. The fox is policing the henhouse.
Discussion: The best part of any article is the discussion, because it is a marketing exhibit.
The results of this placebo-controlled fixed-dose study indicate that aripiprazole impacts beneficially on psychotic symptoms in institutionalized patients suffering from psychosis associated with AD; this is the first such study to report improvement in both primary and secondary outcome measures in this patient population...
Aripiprazole also produced significant improvement in other aspects of psychological and behavioral symptoms, as evident from changes in secondary outcome scores...
Thus, the results indicate that the use of aripiprazole cannot only alleviate the specific symptoms of psychosis, but can also reduce the overall psychological and behavioral burden of AD...
Interestingly, the 5 and 10 mg/day doses of aripiprazole show robust efficacy to improve the CMAI scores from baseline to endpoint, reinforcing the evidence that atypical antipsychotics are efficacious in the treatment of agitation. [Apparently “robust efficacy” means having a miniscule impact relative to placebo.]
Again, see above commentary about the extremely small extent to which Abilify “impacts beneficially” upon participants in this study.
To the authors’ credit, they mention that
Elderly patients with dementia-related psychosis treated with atypical antipsychotic drugs are at an increased risk of death compared to placebo.
Right, kind of like what happened in this study, at least for the patients who took 10 mg of Abilify. I realize it may have been a chance occurrence that the Abilify 10 mg patients died at a higher rate, but when you combine minimal treatment benefit with doubling the risk of death, why the hell would anyone choose to prescribe Abilify based on results from this study?
Believe it or not, there are a couple of other points about the study that I could prattle on about, but I think the point has been made.
Journal Policies: If you wander over to the American Journal of Geriatric Psychiatry’s Instructions for Authors, you can see the following:
Provide measures of effect size liberally. Give cautions when statistical significance has doubtful clinical or practical significance.
What I have done in this post is provide the measures of effect size since the authors did not provide a single measure of effect size. Not a single one. But wait, if this paper clearly violates one of the main principles listed under “Statistic Guidelines” for the Instructions to Authors, how did it get published? Good question. And that takes me right back to my earlier posts on peer review and how the “experts” who review papers sometimes do so in a slipshod, lazy, biased, and/or incompetent fashion (1, 2, 3, 4). And the editor? Did he even skim this manuscript to see if it conformed to the guidelines of the journal?
So a paper that flouts the journal’s own policies is published, a paper in which statements regarding a product’s efficacy are far overblown. Perhaps drug reps will be disseminating this wonderful piece of science nationwide or even internationally to help get patients onto Abilify because now their marketing is based on "science." Will physicians see through this sort of ruse and laugh the reps out of the office, or will they briefly scan the abstract, conclude the drug is effective, and then whip out their prescription pads? You tell me.Last, but not least, I present a Golden Goblet Award to the academic authors of the study as well as to the coauthors at Bristol-Myers Squibb. Your ability to present teeny treatment effects as "robust" and get away with it is notable. Great work. You deserve a good cut of however many $$$ are added to BMS's coffers from unneeded prescriptions of Abilify for dementia.