A Radical Proposal to Address the Problem of the Lack of Generalizability of Placebo-Controlled Studies of Antidepressants

See article by Blanco et al

This work may not be copied, distributed, displayed, published, reproduced, transmitted, modified, posted, sold, licensed, or used for commercial purposes. By downloading this file, you are agreeing to the publisher’s Terms & Conditions.

A Radical Proposal to Address the Problem of the Lack of Generalizability of Placebo-Controlled Studies of Antidepressants

Mark Zimmerman, MDa,*

In the current issue of the Journal, Blanco and colleagues1 examine the generalizability of neuroimaging studies of 5 psychiatric disorders and find that most studies select individuals with "pure" forms of the disorder of interest and exclude the large percentage of individuals with comorbid disorders. The authors suggest that the sample selection procedures of these studies limit the applicability of this research toward understanding the pathophysiology of the patients seen in clinical practice (most of whom do not have pure forms of disorders).

In the introduction of their article, Blanco and colleagues1 note that concerns about generalizability, or external validity, have previously been raised with regard to treatment studies of psychiatric disorders. Their article thus expands the scope of concerns about generalizability, and they most likely foreshadow a future debate about whether the empirical literature is sufficiently robust and generalizable to incorporate and pay for the testing of biomarkers in routine clinical practice.

In present day clinical practice, a more salient concern than problems with the generalizability of neuroimaging studies is the limited external validity of treatment studies. I will therefore use the invitation to write this commentary as an opportunity to make a radical proposal that could nudge the industry toward conducting treatment studies that have greater clinical applicability.

During the past decade, my clinical research group has studied the generalizability of antidepressant efficacy trials (AETs).2-5 We found that most depressed patients seen in our outpatient practice would not have qualified for an AET,3 a result that has been independently replicated a number of times.6-8 Thus, for most patients seen in clinical practice, we do not know if medication works. Of course, many of our patients get better after initiating antidepressants, but we do not know how often this is due to the nonspecific therapeutic aspects of treatment rather than the direct chemical action of the antidepressant molecule. Demonstrating an active medication is superior to placebo has proven to be difficult, and strategies have been discussed and proposed to increase the likelihood of "detecting a signal."9,10 However, as a practicing psychiatrist, I am humbled to realize that most of the patients I treat would not qualify for an AET; therefore, most of my prescriptions of antidepressants are predicated upon "a leap of faith."

The only approach I can envision that will motivate the pharmaceutical industry to change its strategy of recruiting a highly selected group of patients into treatment studies is to impose a countervailing force that would impact revenues. Specifically, if health insurance companies limited prescriptions to patients upon whom antidepressants have proven effective, then I would predict that physician behavior would change abruptly and dramatically, and the resulting financial consequence of this change would alter how patients are recruited into industry-funded AETs. Health insurance companies can easily accomplish this change in prescribing habits by modifying the information required on medication preauthorization forms.

The primary purpose of medication preauthorization is to reduce cost by limiting the prescribing of expensive brand name medications. To be sure, I do not like completing preauthorization forms. As a busy psychiatrist who sees many patients and prescribes many medications, I find the completion of preauthorization forms to prescribe certain medications to be an annoyance. Preauthorization is most commonly required for medications that are not yet available in generic equivalents, and the preauthorization forms are usually limited to 2 questions—one about prior failed treatment efforts (with generic medications) and the other to verify that the patient has the diagnosis for which the medication is indicated. The latter question is designed to limit the off-label use of medication.

Antidepressants are one of the most frequently prescribed classes of medications,11 and most antidepressants are available in generic formulations. Yet during the past 8 years, 4 medications (vortioxetine, vilazodone, desvenlafaxine, and levomilnacipran extended release) have received approval from the US Food and Drug Administration (FDA) for the treatment of major depressive disorder (MDD), and generics are not yet available. The question I would like to raise herein is whether insurance companies would be justified in expanding the information requested on a preauthorization form to include questions reflecting the inclusion and exclusion criteria used to select patients into the AETs that established the medications’ regulatory approval. Another way of putting this—Should insurance companies limit authorization to the patient subgroup for whom the medication has been demonstrated to be effective? I would argue that the FDA has failed to follow its own guidelines regarding the proper labeling of antidepressants and therefore insurance companies would be justified in going beyond current preauthorization procedures in limiting the prescription of costly branded antidepressant medications.

According to the FDA’s Code of Federal Regulations on the labeling of medications (21CFR201.57),12 a product’s label should identify the subgroups of patients for which a medication is effective if there is evidence that the medication’s effectiveness is limited to select subgroups. A separate FDA industry guidance monograph on the labeling of prescription drugs proscribes the content and format of a product’s package insert. The clinical studies section of a product’s label is supposed to identify the important limitations of the empirical evidence supporting a product’s efficacy.13 Specifically, the FDA’s guideline states that a label’s description of the study population "should identify those characteristics that are important for understanding how to interpret and apply the study results. The description thus should identify important inclusion and exclusion criteria’ ¦. For example, the description should discuss enrollment factors that exclude subjects prone to adverse effects, the age distribution of the study population, a baseline value that results in a study population that is more or less sick than usual‘ ¦" [italics added].12

We recently reviewed the inclusion/exclusion criteria of AETs published during the past 20 years.5 We found that all 170 studies included in the review required a minimum score on a depression symptom severity scale.5 The symptom severity inclusion criterion has the greatest impact on the number of patients in routine clinical practice who would not qualify for a study.3 However, no label for an FDA-approved antidepressant includes the caveat that the medication was found to be effective only in patients scoring above a symptom severity cutoff. Thus, the FDA labeling guidelines, which specifically indicate that the routine exclusion of patients of a certain level of severity should be noted in the product’s label, have been consistently ignored. Instead, the product labels of antidepressants indicate approval for the treatment of MDD and not for the treatment of MDD with a minimum level of severity. The FDA has not followed its own guidelines.

Failure to note the lack of generalizability due to the exclusion of less severely depressed patients is particularly noteworthy for the most recently approved medications because the symptom severity inclusion thresholds have been even higher than the cutoffs used in older studies.5 For example, we reviewed the placebo-controlled studies of vortioxetine, the most recently FDA-approved antidepressant.14 Across 12 studies, the mean score at baseline on the Montgomery-Asberg Depression Rating Scale (MADRS)15 ranged from 29.8 to 34.1—in the severe range of severity.16 Thus, mildly depressed patients, who nonetheless met the DSM-IV diagnostic criteria for MDD, and many patients with MDD of moderate severity, would not have been included in any of the 12 vortioxetine studies. While there was some variability between studies in the minimum MADRS score required for inclusion, 11 of the 12 studies required a minimum score of at least 26. The only study17 to use a lower symptom severity inclusion score (22 on the MADRS) failed to find a significant difference between vortioxetine and placebo. It is likely that the shift to requiring higher baseline severity scores is the pharmaceutical industry’s response to the literature suggesting that the difference in efficacy between antidepressants and placebo is greater in more severely ill patients and of uncertain clinical significance in the less severely depressed patients.18-20 Whatever the reason for the higher severity inclusion threshold, the fact is that there is no evidence that vortioxetine is an effective antidepressant for the majority of depressed patients (ie, those scoring below 26 on the MADRS).

To be sure, the exclusion of patients with insufficient symptom severity (despite meeting DSM criteria for MDD) is not the only important inclusion/exclusion criterion limiting the generalizability of AETs. The other frequent exclusion criterion is the presence of a comorbid psychiatric disorder. Again using the vortioxetine studies as an example, every study excluded patients with any comorbid psychiatric disorder.14 The majority of depressed patients in routine clinical practice have a comorbid disorder,21 and the presence of a comorbid disorder predicts poorer outcome.22 Thus, we essentially do not know if vortioxetine is effective for the vast majority of patients with MDD who are treated in routine clinical practice. If the FDA had required the pharmaceutical industry to follow the FDA labeling guidelines, then the vortioxetine label should have read something like this: Vortioxetine is indicated for the treatment of MDD in patients without a comorbid psychiatric disorder and with a minimum severity score of 26 on the Montgomery-Asberg Depression Rating Scale.

It is not surprising that the pharmaceutical industry seeks to recruit patients that are most likely to demonstrate a significant difference between active drug and placebo. I do not blame the industry, as there is no downside to this practice. It is disappointing that the FDA has not enforced its own guidelines regarding the proper labeling of antidepressants. It is likely that drug companies will continue to look for methods to enhance drug-placebo differences, regardless of the limits they impose on generalizability. The question is whether the health insurance industry will step in where the FDA has not. If I were responsible for controlling health care expenditures by limiting (but not eliminating) access to more costly medications, I would restrict medication to those patients in whom it was demonstrated to be effective. That is, I would modify the preauthorization form for vortioxetine to require an attestation that the patient did not have a comorbid psychiatric condition and scored at least 26 on the MADRS. I would predict that prescribing habits would dramatically change if psychiatrists were required to complete a more burdensome (though evidence-based) preauthorization form.

My goal in writing this commentary is not to limit the prescription of antidepressant medications (or any other medications for that matter). Rather, I am hoping that the pharmaceutical industry will change their research methods to be more inclusive and relevant to clinical practice when studying the efficacy of new medications. The prescription of antidepressants has significantly increased over the past 20 years,23 and concerns about their overprescription have been raised.24 It is important to better characterize the depressed patients who are and are not responsive to antidepressant medication and to limit prescribing only to those patients who show greater benefit than that achieved when receiving placebo. Since I am pessimistic that the FDA will suddenly start to follow their own guidelines regarding the labeling of medications, I am suggesting that the health insurance industry, which has a financial stake, step in and enact a change to their preauthorization procedures thereby reducing expenditures on treatments that have not yet been shown to be effective for most depressed patients. I am not implying that the medications are ineffective for these large subgroups of depressed patients. Rather, because of the sample selection procedures in AETs, we simply do not know if the medications are effective in most depressed patients treated in clinical practice. Such a change in the rigor (or should I say burden) of the preauthorization process for recently approved, branded medications would most likely prompt the pharmaceutical industry to conduct studies that have greater external validity because revenues and revenue projections would be negatively impacted.

Submitted: January 25, 2016; accepted January 26, 2016.

Potential conflicts of interest: Dr Zimmerman has been on advisory boards for F Hoffmann-La Roche, Genentech, and Lundbeck; has received research support from Azevan and Eli Lilly; and has prepared educational material for Otsuka.

Funding/support: None.

REFERENCES

1. Blanco C, Wall MM, Lindquist MA, et al. Generalizability of neuroimaging studies in 5 common psychiatric disorders based on the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC). J Clin Psychiatry. 2016;77(12):e1618-e1625.

2. Zimmerman M, Chelminski I, Posternak MA. Exclusion criteria used in antidepressant efficacy trials: consistency across studies and representativeness of samples included. J Nerv Ment Dis. 2004;192(2):87-94. PubMed doi:10.1097/01.nmd.0000110279.23893.82

3. Zimmerman M, Mattia JI, Posternak MA. Are subjects in pharmacological treatment trials of depression representative of patients in routine clinical practice? Am J Psychiatry. 2002;159(3):469-473. PubMed doi:10.1176/appi.ajp.159.3.469

4. Zimmerman M, Posternak MA, Chelminski I. Symptom severity and exclusion from antidepressant efficacy trials. J Clin Psychopharmacol. 2002;22(6):610-614. PubMed doi:10.1097/00004714-200212000-00011

5. Zimmerman M, Clark HL, Multach MD, et al. Have treatment studies of depression become even less generalizable? a review of the inclusion and exclusion criteria in placebo controlled antidepressant efficacy trials published during the past 20 years. Mayo Clin Proc. 2015;90(9):1180-1186. PubMed doi:10.1016/j.mayocp.2015.06.016

6. van der Lem R, van der Wee NJ, van Veen T, et al. The generalizability of antidepressant efficacy trials to routine psychiatric out-patient practice. Psychol Med. 2011;41(7):1353-1363. PubMed doi:10.1017/S0033291710002175

7. Wisniewski SR, Rush AJ, Nierenberg AA, et al. Can phase III trial results of antidepressant medications be generalized to clinical practice? a STAR*D report. Am J Psychiatry. 2009;166(5):599-607. PubMed doi:10.1176/appi.ajp.2008.08071027

8. Zetin M, Hoepner CT. Relevance of exclusion criteria in antidepressant clinical trials: a replication study. J Clin Psychopharmacol. 2007;27(3):295-301. PubMed doi:10.1097/JCP.0b013e318058263f

9. Gelenberg AJ, Thase ME, Meyer RE, et al. The history and current state of antidepressant clinical trial design: a call to action for proof-of-concept studies. J Clin Psychiatry. 2008;69(10):1513-1528. PubMed doi:10.4088/JCP.v69n1001

10. Fava M, Evins AE, Dorer DJ, et al. The problem of the placebo response in clinical trials for psychiatric disorders: culprits, possible remedies, and a novel study design approach. Psychother Psychosom. 2003;72(3):115-127. PubMed doi:10.1159/000069738

11. Mojtabai R, Olfson M. National trends in long-term use of antidepressant medications: results from the US National Health and Nutrition Examination Survey. J Clin Psychiatry. 2014;75(2):169-177. PubMed doi:10.4088/JCP.13m08443

12. US Food and Drug Administration. Code of Federal Regulations Title 21. Department of Health and Human Services: 21CFR20157. US Food and Drug Administration Web site. http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm. 2014. Accessed August 10, 2016.

13. US Food and Drug Administration Center for Drug Evaluation and Research, Center for Biologics Evaluations and Research. Clinical studies section of labeling for human prescription drug and biological products—content and format. Guidance for industry. US Department of Health and Human Services; Rockville, MD. US Food and Drug Administration Web site. http://www.fda.gov/RegulatoryInformation/Guidances/ucm127509.htm. Updated January 2006. Accessed August 10, 2016.

14. Zimmerman M, Clark HL, Multach MD, et al. Inclusion/exclusion criteria in placebo-controlled studies of vortioxetine: comparison to other antidepressants and implications for product labeling. J Affect Disord. 2016;190:357-361. PubMed doi:10.1016/j.jad.2015.10.041

15. Montgomery SA, Asberg M. A new depression scale designed to be sensitive to change. Br J Psychiatry. 1979;134(4):382-389. PubMed doi:10.1192/bjp.134.4.382

16. Kearns NP, Cruickshank CA, McGuigan KJ, et al. A comparison of depression rating scales. Br J Psychiatry. 1982;141:45-49. PubMed doi:10.1192/bjp.141.1.45

17. Mahableshwarkar AR, Jacobsen PL, Chen Y. A randomized, double-blind trial of 2.5 mg and 5 mg vortioxetine (Lu AA21004) versus placebo for 8 weeks in adults with major depressive disorder. Curr Med Res Opin. 2013;29(3):217-226. PubMed doi:10.1185/03007995.2012.761600

18. Fournier JC, DeRubeis RJ, Hollon SD, et al. Antidepressant drug effects and depression severity: a patient-level meta-analysis. JAMA. 2010;303(1):47-53. PubMed doi:10.1001/jama.2009.1943

19. Khan A, Leventhal RM, Khan SR, et al. Severity of depression and response to antidepressants and placebo: an analysis of the Food and Drug Administration database. J Clin Psychopharmacol. 2002;22(1):40-45. PubMed doi:10.1097/00004714-200202000-00007

20. Kirsch I, Deacon BJ, Huedo-Medina TB, et al. Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS Med. 2008;5(2):e45. PubMed doi:10.1371/journal.pmed.0050045

21. Zimmerman M, Chelminski I, McDermut W. Major depressive disorder and axis I diagnostic comorbidity. J Clin Psychiatry. 2002;63(3):187-193. PubMed doi:10.4088/JCP.v63n0303

22. Trivedi MH, Rush AJ, Wisniewski SR, et al; STAR*D Study Team. Evaluation of outcomes with citalopram for depression using measurement-based care in STAR*D: implications for clinical practice. Am J Psychiatry. 2006;163(1):28-40. PubMed doi:10.1176/appi.ajp.163.1.28

23. Olfson M, Marcus SC. National patterns in antidepressant medication treatment. Arch Gen Psychiatry. 2009;66(8):848-856. PubMed doi:10.1001/archgenpsychiatry.2009.81

24. Spence D. Are antidepressants overprescribed? yes. BMJ. 2013;346:f191. PubMed doi:10.1136/bmj.f191