Beyond Statistical Significance: An Underrated Suicide Prevention Intervention

See Reply by Berrouiguet et al and article by Vaiva et al

This work may not be copied, distributed, displayed, published, reproduced, transmitted, modified, posted, sold, licensed, or used for commercial purposes. By downloading this file, you are agreeing to the publisher’s Terms & Conditions.

To the Editor: In their recently published report of a randomized trial, Vaiva et al1 found that suicide attempters allocated to a decision-making algorithm for suicide (ALGOS) were 26% less likely to reattempt suicide within 6 months after discharge than controls who received treatment as usual (TAU) (12.8% vs 17.2% relapses, respectively; difference between groups: 4.4%, 95% confidence interval [CI]: −0.7%, 9.0%; relative risk: 0.74; 95% CI, 0.54-1.01). Because the early postdischarge period entails exceptionally high relapse risk,2 the study’s main outcome was the difference in the percentage of reattempts during the first 6 months of follow-up. The observed difference was described as “not significant” at the provided P value of .059 for the complete-case analysis.

We would like to warn against concluding that the intervention was not effective, given that the conflation of “statistical significance” with decision-making is error-prone.3 In addition, notwithstanding concerns about arbitrary P value cutpoints, 2 key aspects of the study design and analysis should be considered when interpreting the results. First, their control intervention, a priority appointment after discharge combined with a referral to an outpatient clinician for follow-up, has proven effective at lowering relapse risk after suicide attempt in comparable contexts with universal health coverage.4 In fact, enhancing follow-up contacts with health providers is considered the single most effective clinical intervention to reduce suicide behaviors.5 Comparing any intervention to a highly effective TAU can be challenging, and yet there was a difference of over 4 percentage points in suicide reattempts among those randomized to ALGOS. Second, the 2 study groups differed substantially in their loss to follow-up (13.6% in ALGOS vs 18.4% in TAU at 13 months, P = .038). Given that treatment engagement is a key component of suicide prevention efforts,6 we cannot rule out the possibility that lost individuals may have higher relapse rates than the observed individuals. If that was the case, observed data would likely underestimate ALGOS effectiveness. Despite this, all reported analyses in the study were conducted in an intention-to-treat (ITT) basis. Notably, ITT analysis tends to yield conservative effect differences between compared interventions, and, in the presence of loss to follow-up or lack of adherence (especially if it affects differentially the studied interventions, as is the case here), there is no guarantee that an ITT approach adequately estimates the clinical effectiveness of the study.7

Two strategies could have enhanced Vaiva and colleagues’ study report and should be considered in the future. From an analytic perspective, long-lasting trials with substantial loss to follow-up can benefit from being analyzed using several different approaches, including not only intention-to-treat, but also “as-treated” and “per-protocol” analyses, where effect estimates can be controlled by differential adherence and other potential post-randomization, time-varying confounders.7,8 Regarding the interpretation of results, an observed difference between study groups should be judged considering several aspects, including effect size, precision of the estimate, and features of the study design, rather than relying solely on statistical testing.3,9

Suicide is a major global health concern. Suicide attempt, its more reliable risk marker, is an increasingly frequent clinical entity that drives substantial burden for health systems and generates an enormous societal impact.10 Consequently, effective interventions aimed at lowering suicidal behaviors are a priority clinical need. Pragmatic clinical trials conducted in real clinical settings, like the study by Vaiva et al,1 usually entail methodological challenges that can lead to dilution of the effect.11 Nevertheless, results from the real world are sought after by policy makers because of their high external validity.11 We believe that, by estimating the effectiveness of a decision-making algorithm that improves clinicians’ ability to support people at high risk of suicide attempt, Vaiva et al are contributing valuably to suicide prevention.

References

1. Vaiva G, Berrouiguet S, Walter M, et al. Combining postcards, crisis cards, and telephone contact into a decision-making algorithm to reduce suicide reattempt: a randomized clinical trial of a personalized brief contact intervention. J Clin Psychiatry. 2018;79(6):17m11631. PubMed CrossRef

2. Meehan J, Kapur N, Hunt IM, et al. Suicide in mental health in-patients and within 3 months of discharge: national clinical survey. Br J Psychiatry. 2006;188(2):129-134. PubMed CrossRef

3. Greenland S, Senn SJ, Rothman KJ, et al. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. 2016;31(4):337-350. PubMed CrossRef

4. Martí­nez-Alés G, Jiménez-Sola E, Román-Mazuecos E, et al. An emergency department-initiated intervention to lower relapse risk after attempted suicide [published online February 14, 2019]. Suicide Life Threat Behav. PubMed CrossRef

5. Fleischmann A, Bertolote JM, Wasserman D, et al. Effectiveness of brief intervention and contact for suicide attempters: a randomized controlled trial in five countries. Bull World Health Organ. 2008;86(9):703-709. PubMed CrossRef

6. Brenner LA, Barnes SM. Facilitating treatment engagement during high-risk transition periods: a potential suicide prevention strategy. Am J Public Health. 2012;102(suppl 1):S12-S14. PubMed CrossRef

7. Hernán MA, Hernández-Dí­az S. Beyond the intention-to-treat in comparative effectiveness research. Clin Trials. 2012;9(1):48-55. PubMed CrossRef

8. Hernán MA, Hernández-Dí­az S, Robins JM. Randomized trials analyzed as observational studies. Ann Intern Med. 2013;159(8):560-562. PubMed

9. Wasserstein RL, Lazar NA. The ASA’s statement on p-values: context, process, and purpose. Am Stat. 2016;70(2):129-133. CrossRef

10. Mathers CD, Stein C, Fat DM, et al. Global Burden of Disease 2000 (version 2): methods and results. Policy Discussion Paper no. 50:88. Geneva, Switzerland: Global Programme on Evidence for Health, World Health Organization; 2002.

11. Patsopoulos NA. A pragmatic view on pragmatic trials. Dialogues Clin Neurosci. 2011;13(2):217-224. PubMed

Gonzalo Martí­nez-Alés, MD, MSca,b

[email protected]

Katherine M. Keyes, PhDa

Enrique Baca-Garcí­a, MD, PhDb,c,d

aDepartment of Epidemiology, Columbia University Mailman School of Public Health, New York, New York

bDepartment of Psychiatry, Universidad Autónoma de Madrid School of Medicine, Madrid, Spain

cDepartment of Psychiatry, Fundación Jiménez Dí­az University Hospital, Madrid, Spain

dDepartment of Psychiatry, Universidad Católica del Maule, Maule, Chile

Published online: June 18, 2019.

Potential conflicts of interest: The authors declare no related financial disclosures.

Funding/support: None.

J Clin Psychiatry 2019;80(4):19lr12814

To cite: Martí­nez-Alés G, Keyes KM, Baca-Garcí­a E. Beyond statistical significance: an underrated suicide prevention intervention. J Clin Psychiatry. 2019;80(4):19lr12814.

To share: https://doi.org/10.4088/JCP.19lr12814

© Copyright 2019 Physicians Postgraduate Press, Inc.