Risk of Suicide Attempt and Suicide Death Following Completion of the Patient Health Questionnaire Depression Module in Community Practice

ABSTRACT

Suicide remains the 10th-ranked cause of death in the United States, accounting for 41,000 deaths in 2013.1 Suicide attempts lead to 600,000 emergency department visits2 and 200,000 hospitalizations3 annually in the United States. Effective suicide prevention will require accurate identification of those at risk. In 2013, the US Preventive Services Task Force found only “minimal evidence suggesting that some screening tools can identify some adults at increased risk for suicide in primary care.” 4(p741)

Since the publication of the Preventive Services Task Force recommendations, we reported5 that the ninth question of the commonly used Patient Health Questionnaire (PHQ) depression module can accurately identify outpatients at increased risk of suicide attempt and death by suicide. Mental health and primary care patients reporting frequent thoughts of death or self-harm were 6 times as likely to attempt suicide and 5 times as likely to die by suicide during the following year compared to those not reporting such thoughts.

Increasing use of the PHQ and similar self-report scales in community practice6-8 creates both a need and an opportunity to understand how self-reported suicidal ideation predicts suicidal behavior. Clinicians using these questionnaires will need guidance regarding the immediate and long-term risk associated with thoughts of death or self-harm. Large health records databases allow us to address that question.

Here, we use data from approximately 500,000 patients in 4 health systems to examine the relationship between self-reported thoughts of death or self-harm and subsequent suicidal behavior. We focus on 3 questions: Are the relationships reported previously from a single health system replicated in this larger and more diverse sample? What is the immediate risk of suicidal behavior among outpatients reporting thoughts of death or self-harm? How does the risk associated with response to PHQ item 9 change over time?

METHOD

Data are drawn from 4 health systems in the Mental Health Research Network (MHRN), a National Institute of Mental Health-funded consortium of research centers based in large integrated health care systems. Across these systems, electronic medical records and insurance claims have been organized in a Virtual Data Warehouse to facilitate population-based research.9 Protected health information remains at each system, but common data definitions and formats facilitate sharing of de-identified data for research. Institutional Review Boards and privacy boards at each health system approved use of de-identified data for this research.

The 4 MHRN health systems contributing data to this report include Group Health Cooperative, HealthPartners, Kaiser Permanente of Colorado, and Kaiser Permanente of Southern California. These health systems recommend routine use of the PHQ in depression care visits and routinely record E-code (or cause-of-injury) diagnoses, allowing ascertainment of suicide attempts. These systems serve a combined population of approximately 5 million patients in California, Colorado, Minnesota, Washington, and Idaho. Patients are generally representative of each system’s geographic area and are enrolled through a mixture of employer-sponsored insurance, individual insurance, capitated Medicare and Medicaid programs, and other state-subsidized low-income programs.

• Reporting frequent thoughts of death or self-harm indicates a significant and sustained increase in risk of suicide attempt and suicide death.
• Prevention efforts should address suicide risk as an enduring vulnerability rather than a short-term crisis.

The PHQ depression module (often referred to as the PHQ-9) begins with the general instruction “Over the last 2 weeks, how often have you been bothered by any of the following problems?” The ninth item asks about “Thoughts that you would be better off dead or of hurting yourself in some way” with response options including “Not at all,” “Several days,” “More than half the days,” and “Nearly every day.”

The study sample included all PHQ results in electronic medical records between January 1, 2007 and December 31, 2012. Any individual patient could contribute multiple PHQ observations. Suicide deaths were identified using cause-of-death data from state death certificate files. Deaths were classified as due to suicide if ICD-10 cause-of-death codes indicated definite (X60 through X84) or possible (Y10 through Y34) self-inflicted injury. Death certificate data were available through December 31, 2012. Nonfatal suicide attempts were identified using electronic medical records (for services provided at health system facilities) and insurance claims (for services provided by external providers or facilities). These data were available through December 31, 2013. Three criteria were used to identify nonfatal suicide attempts:

1. ICD-9 diagnosis of self-inflicted injury or poisoning (E950 through E958)
2. ICD-9 diagnosis of injury or poisoning considered possibly self-inflicted (E980 through E988)
3. ICD-9 diagnosis of suicidal ideation (V62.84) accompanied by a diagnosis of either poisoning (960 through 989) or open wound (870 through 897).

We previously reported5,10,11 that validation by clinician review of 200 full-text medical records found a positive predictive value of 100% for the first criterion, 80% for the second, and 86% for the third (compared to treating providers’ documentation of self-inflicted injury with suicidal intent). Given the relative frequency of these 3 criteria in our sample, we estimate the weighted mean positive predictive value to be 94% for all 3 combined.

The proportion of medically treated suicide attempts that might be missed by this method was estimated by examining the proportion of emergency department and hospital encounters for injury or poisoning in which a cause-of-injury code (or E-code) was not recorded.12 In 2010, the proportion of injury or poisoning encounters with no E-code ranged from 11% to 26% in these 4 health systems.13

Descriptive analyses examined risk of suicide attempt (nonfatal or fatal) and suicide death following completion of a PHQ, stratified by response to item 9. Analyses of long-term risk accounted both for multiple PHQ observations per person and for censoring of availability of outcome data over time. Each new questionnaire defined a new period at risk, and a patient could contribute multiple overlapping risk periods by completing multiple PHQs. Each suicide attempt or death could be linked to multiple prior PHQ results from a single patient. This approach examines risk based on data available when the PHQ was completed, regardless of subsequent questionnaires. It avoids informative censoring that would occur if likelihood of completing a later PHQ was related to risk of a subsequent suicide attempt. For analyses of suicide attempts, each risk period was censored at the time of disenrollment from the health system, death from causes other than suicide, or last availability of suicide attempt data. For analyses of suicide deaths, each risk period was censored at the time of disenrollment from the health system, death from causes other than suicide, or last availability of suicide death data. Because predictors of repeat suicide attempts may differ from first attempts, PHQ-9 observations following a suicide attempt were excluded from analyses of suicide attempts.

Partly conditional Cox proportional hazards regression14 was used to estimate the association between response to PHQ item 9 and subsequent suicide attempt or death after accounting for potential confounders (age, sex, and health system). Estimated hazard associated with responding “nearly every day,” “more than half the days,” and “several days” was compared to estimated hazard associated with responding “not at all” (the reference group). Because each individual patient could contribute multiple PHQ responses, the robust sandwich estimator15 was used to calculate confidence limits for hazard ratio estimates. To examine how the relationship between response to item 9 and hazard of subsequent suicidal behavior did or did not decrease over time, we compared alternative models including either a constant effect of item 9 response over time or 2 possible time-varying effects: a linear decrease or a log-linear decrease over time. Alternative models were evaluated using the Bayesian Information Criterion statistic [BIC]16,17; a decrease of 10 points is considered a meaningful improvement in model fit.

RESULTS

After the previously described eligibility criteria and exclusions were applied, electronic medical records contained 1,228,308 PHQ results for 509,945 patients, including 302,659 patients (59%) with a single questionnaire, 95,880 (19%) with 2, 38,321 (8%) with 3, 20,204 (4%) with 4, and 52,881 (10%) with 5 or more. Encounters in which questionnaires were administered included 500,751 (41%) with specialty mental health providers, 442,509 (36%) with primary care providers, and 285,048 (23%) with various other medical or surgical specialists (including emergency department visits). At time of completing the PHQ, 709,580 (58%) had received some mental health diagnosis or treatment in the previous 90 days. Using the conventional classification of PHQ scores, depression was minimal (score 0 to 4) in 408,575 (33%), mild (score 5 to 9) in 295,907 (24%), moderate (score 10 to 14) in 221,845 (18%), moderately severe (score 15 to 19) in 153,946 (13%), and severe (20 or greater) in 103,098 (8%). As shown in Table 1, patients completing PHQs were predominantly female and were diverse with respect to age and race/ethnicity. Approximately 3.5% reported thoughts of death or self-harm “more than half the days” and approximately 2.4% reported such thoughts “nearly every day,” but these proportions varied according to age and race/ethnicity.

In this sample, death certificate data identified 484 suicide deaths, including 457 attributed to definite self-inflicted injury and 27 attributed to possible self-inflicted injury.

Electronic medical records and insurance claims data identified 9,203 unique instances of nonfatal suicide attempt, limited to the first attempt per person. These included 6,345 diagnoses of definite self-inflicted injury, an additional 2,105 diagnoses of possible self-inflicted injury, and an additional 753 probable suicide attempts identified by the combination of a poisoning or wound diagnosis linked to a diagnosis of suicidal ideation.

Table 2 describes counts and rates of suicide attempts and deaths in 7 days and 30 days after completing the PHQ. While rates of suicide attempt and suicide death increased with escalating responses to PHQ item 9, over one-third of suicide attempts and suicide deaths occurred among those responding “not at all.” Even among those reporting thoughts of death or self-harm nearly every day, risk of any suicide attempt within 7 days was approximately 1 in 500 and risk of suicide death was approximately 1 in 10,000.

The estimated long-term hazard of any suicide attempt (nonfatal or fatal) stratified by response to PHQ item 9 is shown in Figure 1A. Over 2 years, cumulative hazard ranged from approximately 0.5% (1 in 200) for those responding “not at all” to approximately 3.5% (1 in 30) for those responding “nearly every day.”

In a partly conditional Cox proportional hazards model predicting risk of any suicide attempt, addition of item 9 response (3 indicator variables for levels of positive response) was a highly significant predictor of mean risk over 2 years (P < .00001 compared to model including only age, sex, and study site). Allowing the effect of item 9 responses to vary linearly over time significantly improved model fit (BIC decreased from 255,003 to 254,813). Model fit was further improved by allowing the effect of item 9 responses to vary over time in a log-linear fashion (BIC decreased to 254,786). Table 3 displays the estimated relative hazards of suicide attempt from the final model allowing a log-linear change over time. The excess hazard associated with reporting thoughts of death or self-harm did decrease over time, but relative hazards for all positive responses remained significantly elevated through 18 months.

The estimated long-term hazard of suicide death according to item 9 response is shown in Figure 1B. Over 2 years, cumulative hazard ranged from approximately 0.04% (1 in 2,500) for those responding “not at all” to approximately 0.2% (1 in 500) for those responding “nearly every day.” No clear separation was observed between those responding “several days” and “more than half the days.”

In a partly conditional Cox proportional hazards model predicting risk of suicide death by age, sex, and study site, addition of 3 indicator variables for item 9 response was a significant predictor of mean hazard over 2 years (P < .0001). Table 3 displays the estimated relative hazard of suicide death for different responses to PHQ item 9, again allowing the effect of item 9 scores to vary over time in a log-linear fashion. Confidence intervals were wider than those for relative hazard of suicide attempt, reflecting the smaller number of events. Estimated relative hazard associated with a response of “more than half the days” appears to decline more with time than seen for suicide attempts. No statistically significant differences were observed between the “several days” and “more than half the days” groups at any time point. Otherwise, the overall pattern was generally similar to that seen for suicide attempts: relative hazard estimates declined over time but remained significantly elevated through 18 months (see Table 3).

The proportional hazards models described above were used to examine the unique contribution of item 9 to predicting suicidal behavior, over and above the effect of depression severity (as indicated by items 1 through 8). As shown in Table 4, adding categorical indicators for overall depression severity only slightly attenuated the excess hazard associated with response to item 9 for prediction of suicide attempts (column 2 vs column 3) or suicide deaths (column 4 vs column 5).

Analyses stratified by sex found that absolute hazard of suicide risk was higher in men, but relative hazard of suicide attempt and suicide death according to PHQ item 9 response was generally similar for men and women (see eAppendix 1 at PSYCHIATRIST.COM for details).

DISCUSSION

In this sample of over 1.2 million PHQ depression questionnaires completed by over 500,000 outpatients, we find that response to PHQ item 9 was a strong predictor of suicide attempt and suicide death over the following 2 years. These results extend our previous findings in 3 significant ways. First, the strong association between item 9 response and subsequent suicidal behavior is confirmed in a more diverse sample from 4 large health care systems (Figure 1). Second, we can more precisely describe the immediate risk of suicidal behavior, a significant concern for practicing clinicians (Table 2). Third, long-term follow-up data allow us to describe how risk evolves over time (Table 3).

These real-world data have several important strengths. The sample was not limited to research volunteers and included all PHQ responses in a large, diverse sample from community practice. Thoughts of death or self-harm were reported in real time rather than recalled months or years later. Suicide attempts and suicide deaths were recorded independently and prospectively. These results illustrate the potential for “big data” from electronic health records to accelerate research and inform quality improvement regarding suicide risk.

Diagnoses of self-inflicted injury, our primary criterion for identifying suicide attempts, may have included cases of self-inflicted injury without suicidal intent. Our chart review validation, however, found a positive predictive value of over 90% for provider-documented suicidal intent.

While frequent thoughts of death or self-harm were strongly associated with subsequent suicidal behavior, we should emphasize the trade-off between sensitivity (identifying all patients at risk) and positive predictive value (identifying those at highest risk). As shown in Table 2, any positive response to item 9 would have a sensitivity of 61% (698/1145) for detecting any suicide attempt and 64% (50/78) for detecting suicide death over 1 month. But the 2-year cumulative hazard of suicide attempt in those with any positive response is less than 2%. Using a higher threshold of “more than half the days” or “nearly every day” would increase cumulative hazard or positive predictive value to approximately 3% but reduce sensitivity to approximately 35%. This trade-off is not unique to identifying suicide risk; it was described over 30 years ago concerning cardiovascular disease prevention.18 Interventions limited to those at highest risk will not reach the medium-risk population for whom most morbidity and mortality occur. For perspective, a 3% hazard of suicide attempt in outpatients with frequent thoughts of death or self-harm is similar to the hazard of stroke in outpatients with atrial fibrillation and hypertension.19

The excess risk associated with reporting thoughts of death or self-harm did decrease with time. For example, the relative hazard of suicide attempt associated with reporting such thoughts “nearly every day” (compared to responding “not at all”) decreased from approximately 10 to approximately 4 over 18 months. Relative hazard of suicide death showed a similar pattern of decrease over time. Nevertheless, reporting frequent thoughts of death or self-harm was associated with statistically significant and clinically important increases in risk of suicide attempt and suicide death for at least 18 months.

The PHQ was designed to assess depression severity, not to assess suicide risk. Broader questions regarding hopelessness or suicidal thoughts, such as those in the Beck Depression Inventory20 or the Quick Inventory of Depressive Symptomatology,21 might prove more sensitive. Questions focused on suicidal ideation, such as the Columbia Suicide Severity Rating Scale22 or the Scale for Suicidal Ideation,23 might identify a smaller group at higher risk. At this time, we lack prospective data from community practice regarding those alternative assessments. We focus on the PHQ not because it is the optimal measure, but because it is widely used in community practice.

Most questionnaires in this sample were completed by patients receiving mental health treatment. Questions regarding thoughts of death or self-harm might have different meaning for patients with no mental health history. Our sample includes too few questionnaires from emergency departments or other specialty clinics to examine whether results differ in those specific settings. In most cases, treating clinicians were aware of PHQ responses and might be expected to take appropriate clinical actions to reduce risk. The association we observe between item 9 response and subsequent risk occurred in spite of any such clinical intervention.

Our findings certainly do not suggest that clinicians can use the PHQ or any other assessment to accurately predict short-term risk of suicidal behavior. Instead, we find that short-term prediction is not feasible and that emphasis on short-term risk is misplaced. Frequent thoughts of death or self-harm indicate an enduring vulnerability rather than simply a short-term crisis.

Any efforts to identify and reduce risk of suicidal behavior must be guided by evidence regarding risk levels in different populations. In the general population, for whom 1-year risk of suicide attempt is less than 1 per 1,000, community-level programs can increase public awareness or reduce availability of lethal means.24 At the other extreme, intensive programs25,26 focus on the small group of patients with recent or repeated suicide attempts, for whom 1-year risk approaches 1 in 2. Our findings suggest that responses to the commonly used PHQ depression scale identify a large intermediate group for whom 2-year risk of suicide attempt exceeds 1 in 30. Additional research is needed to identify effective interventions appropriate for this at-risk group.

The Preventive Services Task Force recommendation against screening for suicide risk cited the absence of data regarding either the accuracy of screening tests or the effectiveness of preventive interventions. Our findings address only the first of those gaps. While our data cannot assess the effectiveness of interventions, they do provide guidance regarding timing of prevention programs. Among outpatients reporting thoughts of self-harm “nearly every day,” cumulative hazard of suicide attempt was approximately 1 in 500 after 1 week, increasing to 1 in 30 over 2 years. Hospitalization or crisis-focused interventions may sometimes be necessary, but they cannot address risk that endures for months or years. Sustained interventions, most likely focused on skill development and maintaining treatment engagement, will be necessary to address sustained risk.

Submitted: January 5, 2015; accepted May 28, 2015.

Disclosure of off-label usage: The authors have determined that, to the best of their knowledge, no investigational information about pharmaceutical agents that is outside US Food and Drug Administration-approved labeling has been presented in this article.

Financial disclosure: Drs Simon, Coleman, Rossum, Beck, Whiteside, Penfold, Shortreed, and Rutter, Ms Oliver, Mr Johnson, and Ms Operskalski have no personal affiliations or financial relationships with any commercial interest to disclose relative to the article.

Funding/support: Supported by Cooperative Agreement U19MH092201 with the National Institute of Mental Health.

Role of the sponsor: The funder (National Institute of Mental Health) was not involved in the design of the research, data analysis, interpretation, or preparation of this article.

Acknowledgments: Dr Simon had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Original data in the HMO Research Network (HMORN) Virtual Data Warehouse are owned by and remain under the control of the 4 participating health care systems. Investigators are allowed to access these data to address specific research questions, subject to specific approval by each health system’s Institutional Review Board and Privacy Board.

Supplementary material: Available at PSYCHIATRIST.COM.

REFERENCES

1. National Center for Health Statistics. Deaths: Final Data for 2013. Hyattsville, MD: National Vital Statistics Reports; 2015.

2. Centers for Disease Control and Prevention. National Hospital Ambulatory Medical Care Survey: 2008 Emergency Department Summary Tables Atlanta. Centers for Disease Control and Prevention Web site. http://www.cdc.gov/nchs/data/ahcd/nhamcs_emergency/nhamcsed2008.pdf. Updated January 6, 2012; Accessed February 22, 2012.

3. Crosby AE, Han B, Ortega LAG, et al; Centers for Disease Control and Prevention (CDC). Suicidal thoughts and behaviors among adults aged ≥18 years—United States, 2008-2009. MMWR Surveill Summ. 2011;60(13):1-22. PubMed

4. O’ Connor E, Gaynes BN, Burda BU, et al. Screening for and treatment of suicide risk relevant to primary care: a systematic review for the US Preventive Services Task Force. Ann Intern Med. 2013;158(10):741-754. PubMed doi:10.7326/0003-4819-158-10-201305210-00642

5. Simon GE, Rutter CM, Peterson D, et al. Does response on the PHQ-9 Depression Questionnaire predict subsequent suicide attempt or suicide death? Psychiatr Serv. 2013;64(12):1195-1202. PubMed doi:10.1176/appi.ps.201200587

6. Harding KJ, Rush AJ, Arbuckle M, et al. Measurement-based care in psychiatric practice: a policy framework for implementation. J Clin Psychiatry. 2011;72(8):1136-1143. PubMed doi:10.4088/JCP.10r06282whi

7. Kendrick T, Dowrick C, McBride A, et al. Management of depression in UK general practice in relation to scores on depression severity questionnaires: analysis of medical record data. BMJ. 2009;338:b750. PubMed doi:10.1136/bmj.b750

8. Valuck RJ, Anderson HO, Libby AM, et al. Enhancing electronic health record measurement of depression severity and suicide ideation: a Distributed Ambulatory Research in Therapeutics Network (DARTNet) study. J Am Board Fam Med. 2012;25(5):582-593. PubMed doi:10.3122/jabfm.2012.05.110053

9. Ross TR, Ng D, Brown JS, et al. The HMO Research Network Virtual Data Warehouse: a public data model to support collaboration. EGEMS (Wash DC). 2014;2(1):1049. PubMed

10. Simon GE, Savarino J, Operskalski B, et al. Suicide risk during antidepressant treatment. Am J Psychiatry. 2006;163(1):41-47. PubMed doi:10.1176/appi.ajp.163.1.41

11. Simon GE, Savarino J. Suicide attempts among patients starting depression treatment with medications or psychotherapy. Am J Psychiatry. 2007;164(7):1029-1034. PubMed doi:10.1176/ajp.2007.164.7.1029

12. Patrick AR, Miller M, Barber CW, et al. Identification of hospitalizations for intentional self-harm when E-codes are incompletely recorded. Pharmacoepidemiol Drug Saf. 2010;19(12):1263-1275. PubMed doi:10.1002/pds.2037

13. Lu CY, Stewart C, Ahmed AT, et al. How complete are E-codes in commercial plan claims databases? Pharmacoepidemiol Drug Saf. 2014;23(2):218-220. PubMed doi:10.1002/pds.3551

14. Zheng Y, Heagerty PJ. Partly conditional survival models for longitudinal data. Biometrics. 2005;61(2):379-391. PubMed doi:10.1111/j.1541-0420.2005.00323.x

15. Lin DY, Wei LJ. The robust inference for the proportional hazards model. J Am Stat Assoc. 1989;84(408):1074-1078. doi:10.1080/01621459.1989.10478874

16. Kass RE, Raftery AE. Bayes Factors. J Am Stat Assoc. 1995;90(430):773-795. doi:10.1080/01621459.1995.10476572

17. Volinsky CT, Raftery AE. Bayesian information criterion for censored survival models. Biometrics. 2000;56(1):256-262. PubMed doi:10.1111/j.0006-341X.2000.00256.x

18. Rose G. Strategy of prevention: lessons from cardiovascular disease. Br Med J (Clin Res Ed). 1981;282(6279):1847-1851. PubMed doi:10.1136/bmj.282.6279.1847

19. Singer DE, Albers GW, Dalen JE, et al; American College of Chest Physicians. Antithrombotic therapy in atrial fibrillation: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (8th Edition). Chest. 2008;133(6 suppl):546S-592S. doi:10.1378/chest.08-0678 PubMed

20. Beck AT, Beamesderfer A. Assessment of depression: the depression inventory. Mod Probl Pharmacopsychiatry. 1974;7(0):151-169. PubMed doi:10.1159/000395074

21. Rush AJ, Trivedi MH, Ibrahim HM, et al. The 16-Item Quick Inventory of Depressive Symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): a psychometric evaluation in patients with chronic major depression. Biol Psychiatry. 2003;54(5):573-583. PubMed doi:10.1016/S0006-3223(02)01866-8

22. Posner K, Brown GK, Stanley B, et al. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. 2011;168(12):1266-1277. PubMed doi:10.1176/appi.ajp.2011.10111704

23. Beck AT, Brown GK, Steer RA, et al. Suicide ideation at its worst point: a predictor of eventual suicide in psychiatric outpatients. Suicide Life Threat Behav. 1999;29(1):1-9. PubMed

24. Mann JJ, Apter A, Bertolote J, et al. Suicide prevention strategies: a systematic review. JAMA. 2005;294(16):2064-2074. PubMed doi:10.1001/jama.294.16.2064

25. Linehan MM, Comtois KA, Murray AM, et al. Two-year randomized controlled trial and follow-up of dialectical behavior therapy vs therapy by experts for suicidal behaviors and borderline personality disorder. Arch Gen Psychiatry. 2006;63(7):757-766. PubMed doi:10.1001/archpsyc.63.7.757

26. Brown GK, Ten Have T, Henriques GR, et al. Cognitive therapy for the prevention of suicide attempts: a randomized controlled trial. JAMA. 2005;294(5):563-570. PubMed doi:10.1001/jama.294.5.563