Effect of Scheduling a Post-Discharge Outpatient Mental Health Appointment on the Likelihood of Successful Transition From Hospital to Community-Based Care

ABSTRACT

The period immediately following discharge from inpatient psychiatric care poses substantial risks of serious and even life-threatening adverse outcomes. Foremost among them is the risk of premature treatment disengagement,1-4 which in turn increases the risk of relapse and hospital readmission,5-11 homelessness,12,13 violent behavior,14,15 criminal justice involvement,16,17 and all-cause mortality including suicide.18-20 These risks are especially concerning given the high rates of failed transitions from inpatient to outpatient mental health care: 42%-51% of adults1,2,21 and 31%-45% of youth21-24 do not receive any outpatient mental health treatment for their disorders within 30 days of inpatient discharge.

Scheduling timely appointments for outpatient mental health follow-up care is widely accepted as a standard of care for inpatient psychiatric treatment.25-30 Little is known, however, regarding the proportions of hospitalized individuals who receive this routine discharge planning activity as well as its effectiveness. Timely scheduling of initial outpatient visits following discharge has been associated with improved rates of attending outpatient mental health services,31-37 although negative findings have also been reported.38 However, nearly all prior studies were either single-site reports or did not rigorously control for differences between patients who did and did not receive scheduling of outpatient mental health appointments.4,32,39,40

In this study, we examined the association between having an outpatient mental health appointment scheduled as part of discharge planning and attending outpatient mental health care, controlling for several patient, hospital, and system characteristics in a cohort of over 15,000 inpatient psychiatric admissions. Additionally, we repeated the analyses for the subgroup of patients who had co-occurring substance use disorders, a patient population known to be especially unlikely to receive comprehensive discharge planning and at high risk for failed care transitions.41 For both analyses, we hypothesized that after control for potential confounds, patients who had an outpatient appointment scheduled as part of discharge planning would have a higher likelihood of attending an initial outpatient mental health service.

METHODS

Data were collected as part of a statewide quality assurance program and were obtained from 4 primary sources: (1) Medicaid claims records (including both patient and clinician data), (2) the 2012-2013 American Hospital Association Annual Survey, (3) the 2012-2013 Health Resources and Human Services Administration Area Resource File, and (4) a 2012-2013 New York State (NYS) Managed Behavioral Healthcare Organization (MBHO) Discharge File created during a statewide quality assurance program in which NYS contracted with 5 MBHOs in geographically distinct regions to review discharge planning practices for fee-for-service inpatient psychiatric admissions. Medicaid claims and the MBHO Discharge File data are owned by the NYS Department of Health and Office of Mental Health, respectively, and are not available for sharing. Area Resource File data are available from the federal Health Resources and Human Services Administration, and annual hospital survey data can be obtained from the American Hospital Association.

Patients were eligible for the initial sample if they (1) were < 65 years of age, (2) were admitted to an inpatient psychiatric unit during 2012-2013 with a principal diagnosis of a mental disorder, (3) had an inpatient length of stay of ≤ 60 days, (4) were discharged to the community, (5) had Medicaid eligibility for at least 11 of the 12 months prior to admission, and (6) were not Medicare beneficiaries. A total of 18,793 subjects met these criteria. Of these, 888 patients did not have information relating to outpatient appointments made as a part of the discharge plan, yielding a population of 17,905 subjects considered for propensity score adjustment (described subsequently). Only the first observed inpatient admission was included for patients with more than one inpatient psychiatric admission during 2012-2013. The local Institutional Review Board approved the study and granted a waiver of individual consent.

Variables of Interest

The outcome variables, which were calculated with Medicaid claims data, were defined as attending an outpatient mental health service within 7 and 30 days following inpatient discharge. An outpatient mental health service visit was defined as a Medicaid claim for a visit at a state mental health-licensed outpatient setting or any outpatient service with a primary diagnosis of a mental disorder provided by a mental health practitioner or physician. Outpatient visits were tracked for up to 60 days following hospital discharge, under the assumption that care transition practices were unlikely to affect longer-term outpatient follow-up care.

The primary independent variable was a dichotomous measure of whether the inpatient psychiatric treatment team scheduled a mental health outpatient appointment following discharge. This variable was reported for each discharge by the MBHOs, which were instructed to code whether an appointment for outpatient mental health treatment was part of the discharge plan for each case and, if so, to list the date of the appointment. In a reliability study, 214 randomly selected hospital medical records of patients included in the cohort were compared to MBHO ratings. Raters reviewing the hospital records were required to verify that the aftercare provider listed in the medical record was a licensed mental health clinic, mental health specialty provider (eg, Assertive Community Treatment), or behavioral health independent practitioner. Using these rating guidelines for the hospital medical records, we found acceptable reliability (κ = 0.43) between the MBHO reports of appointments being scheduled and documentation in the hospital medical records.

Propensity Score Adjustment and Analytic Plan

Given the observational nature of the data, we expected an imbalance among the patient, hospital, and service system characteristics of inpatients who did and did not have outpatient appointments scheduled as part of their discharge plan. Accordingly, among the 17,905 eligible subjects, propensity scores were created to account for the likelihood of inpatient providers’ scheduling an outpatient appointment. Propensity scores estimated the probability of having an outpatient mental health appointment scheduled conditional on observed patient, hospital, and service system factors.

Patient-level variables included demographic characteristics, primary inpatient discharge diagnosis, co-occurring substance use disorder at discharge, prior engagement in outpatient mental health services preceding admission, homelessness at admission, and burden of co-occurring medical conditions using a modified Elixhauser Comorbidity Index (ECI) that calculated scores based on clinical diagnoses recorded in inpatient or outpatient records during the 12 months prior to inpatient admission.42-44 The ECI score was modified to exclude mental disorder categories (depression, psychosis, alcohol use, and drug use) because all of the patients in the study population had a mental disorder diagnosis and we sought a measure capturing only non-psychiatric medical comorbidity.

• Scheduling a follow-up outpatient mental health appointment is a widely recommended discharge planning practice for inpatient psychiatric care, yet little is known regarding how often patients receive this practice as well as its effectiveness.
• Scheduling an outpatient appointment increased the odds of patients attending a post-discharge appointment after controlling for a wide range of patient, hospital, and service system characteristics known to impact care transitions.
• Even among patients at highest risk for failed care transitions—eg, those who were homeless, had co-occurring substance use disorders, or were not engaged in care prior to admission—scheduling an aftercare appointment significantly increased the odds of a successful care transition.

Hospital-level variables included number of hospital beds, hospital ownership, psychiatric discharges, psychiatric discharges with substance use disorder diagnosis, percentage of psychiatric population with 2 or more psychiatric discharges, medical resident teaching status, and whether the hospital provided outpatient mental health services. Finally, service system variables based on counties in which patients resided documented regional mental health resources (provider density), poverty, and urban/rural regions. An additional variable was created to distinguish among the 5 different MBHOs reporting data in the study.

Using all 20 patient-, hospital-, and system-level variables, random forest regression was employed to construct the propensity scores for the likelihood of having an outpatient mental health appointment scheduled. Tuning parameters were optimized using the out-of-bag (OOB) misclassification rate with 4 randomly selected variables assessed at each node split, a leaf size of 1, and a maximum tree depth of 50 nodes. The OOB misclassification rate remained stable between 50 and 500 trees, so 50 trees were used. Once propensity scores were calculated, we examined the overlap of the distribution of propensity scores for those with and without appointments scheduled; 2,385 subjects exhibited propensity scores that were non-overlapping, ie, being either higher or lower than all scores in the opposing group, and were thus trimmed from the sample. Scores for the remaining 15,520 subjects were split into 5 strata of equal width. Standardized differences for all 20 independent variables were averaged across the 5 strata as a measure of achieved balance (see Table 1).

Kaplan-Meier curves to visualize differences in outpatient care attendance over time between the two groups were used. Logistic regression models estimated the association between having an outpatient appointment scheduled and 7- and 30-day attendance at outpatient mental health services. Both logistic regression models were fit with generalized estimating equations to control for the within-hospital correlation of observations and adjusted for propensity score stratum. We also fit models by adding an interaction term to test whether propensity score stratum influenced the association between having an appointment scheduled and attending outpatient care. Finally, the full analysis was repeated for those subjects with a co-occurring discharge diagnosis of substance use disorder.

All analyses were performed using SAS version 9.4 (2016; SAS Institute Inc; Cary, North Carolina), including PROC HPFOREST in SAS Enterprise Miner for random forest regression.

RESULTS

The final analytic sample included 15,520 unique individuals discharged from inpatient psychiatric units to the community during the 2-year study period for whom information was available regarding whether a post-discharge appointment was scheduled prior to their discharge. Among these discharged patients, 11,945 (77%) had an outpatient appointment scheduled with a mental health provider as part of their discharge planning. Of the total sample, 5,958 (38%) attended an outpatient mental health service within 7 days following discharge and 9,146 (59%) attended a service within 30 days of discharge.

Figure 1 shows the cumulative rate of patients attending outpatient mental health visits following discharge, with separate curves displaying cumulative probabilities for those who had and did not have an outpatient mental health appointment scheduled as part of their discharge plan. The differences between the two groups was evident by the second day following discharge and increased in magnitude until around day 10 following discharge; after day 10, the difference in cumulative probabilities of attending outpatient visits between those who did or did not have appointments scheduled appeared relatively constant (Figure 1). The proportions of patients attending outpatient visits within 7 and 30 days were 23% and 39%, respectively, for patients who did not have appointments scheduled as part of their discharge plans, versus 43% and 65%, respectively, for patients who did have appointments scheduled prior to discharge.

Patient-, hospital-, and system-level characteristics of the 15,520 patients are presented in Table 1. For propensity score stratification, standardized differences were calculated comparing patients who did and did not have an outpatient mental health appointment scheduled as part of their discharge plan. After trimming cases to create the final propensity score strata and optimally minimize residual bias,45 only 2 variables had standardized difference scores > 0.2 (Table 1): homelessness status at hospital admission and MBHO completing the review. We additionally controlled for those 2 variables in a sensitivity analysis, and the results did not change.

Table 2 summarizes results of unadjusted and adjusted logistic regression models for the association between having an outpatient appointment scheduled as part of discharge planning and attending an outpatient mental health service within 7 and 30 days following discharge. In the propensity score-adjusted model, patients who had an outpatient appointment scheduled as part of their discharge plan were significantly more likely to attend outpatient services within 7 days (OR = 1.69; 95% CI, 1.48-1.94; P < .0001) and 30 days (OR = 1.65; 95% CI, 1.42-1.93; P < .0001) of hospital discharge as compared to those who did not have an appointment scheduled.

Figure 2 displays results for each of the 5 propensity score strata at 7 and 30 days, respectively. Individuals who had an outpatient mental health appointment scheduled as part of their discharge plan had a greater odds of attending an outpatient mental health service following discharge regardless of the probability of having been scheduled for an appointment. In Figure 2, even in propensity score stratum 1, representing patients with the highest concentration of characteristics associated with not having an outpatient appointment scheduled, having an appointment scheduled increased the odds of attending an outpatient visit within 7 days. This association was significant for all strata except for stratum 1, in which it failed to achieve a P value < .05 in part because of the small sample size. This stratum included 387 patients who did not have an outpatient mental health appointment scheduled as part of their discharge plans and 29 patients who had an outpatient appointment scheduled.

The propensity score analysis was repeated for the subgroup of discharged patients with a co-occurring substance use diagnosis at hospital discharge (n = 5,744). Of these patients, 1,626 (28%) and 2,697 (47%) attended an outpatient visit within 7 and 30 days, respectively. In the adjusted model for this subgroup, patients who had an outpatient appointment scheduled with a mental health provider as part of their discharge plan were more likely to attend outpatient mental health services within 7 days (34% attended a visit; OR = 2.05; 95% CI, 1.68-2.51; P < .0001) and 30 days (54% attended a visit; OR = 1.83; 95% CI, 1.52-2.2; P < .0001) as compared to those who did not have an appointment scheduled (Table 2). Odds ratios favored patients who had appointments scheduled and were significant at P < .05 for all analyses except for the 7-day outcome in stratum 1, which included patients who were least likely to have an appointment scheduled and was much smaller. This stratum included 40 patients who did not have an appointment scheduled and 5 who did have an appointment scheduled.

DISCUSSION

Scheduling follow-up mental health appointments is an accepted standard of care for inpatient psychiatric discharge planning, and there is no rationale for not completing this routine practice. There is, however, little research documenting the prevalence of this practice and its impact on care transitions. These questions are especially relevant given the substantial proportions of patients who fail to attend outpatient mental health care in a timely manner following discharge from inpatient psychiatric units: 41% of discharged patients in this study did not see an outpatient mental health provider in the 30 days following discharge, similar to in other published reports.1,2 Discharge planning practices that promote successful care transitions should be key focus areas for clinical quality measurement and improvement activities.

We had a unique opportunity to evaluate the association between this discharge planning activity and successful care transitions by linking multiple administrative databases for a large cohort. Psychiatric inpatient teams scheduled outpatient follow-up appointments with a mental health provider for 77% of more than 15,000 Medicaid beneficiaries discharged from psychiatric inpatient units over a 2-year period. Patients who had a post-discharge appointment scheduled were significantly more likely to attend an outpatient mental health service within both 7 and 30 days following hospital discharge, even after control for a wide range of factors that were correlated with scheduling a follow-up appointment. Our data also indicate that scheduling an appointment as part of the discharge plan is most likely to have an impact on care transitions in the first 7-10 days following discharge. This finding is consistent with known standards of care endorsing scheduling appointments within 7 days following discharge; one would not expect routine discharge planning activities to influence treatment behaviors over longer periods following discharge when other clinical, social, and environmental factors become more likely to have an impact.

Although we could find no published data for comparison, our finding that 77% of discharged patients had an outpatient appointment scheduled as part of their discharge plan appears lower than expected given that this routine practice is a standard of care for hospitalized patients endorsed by opinion leaders,25 the Institute of Medicine,26 the National Committee for Quality Assurance,27 and the Joint Commission.28 There are several reasons why psychiatric inpatient teams may not schedule post-discharge appointments including patients refusing follow-up, lack of community providers able to see patients in a timely manner, and short lengths of stay that strain resources to complete adequate discharge planning. Further research is needed to identify strategies for psychiatric inpatient teams to increase rates of completing routine discharge planning activities for all patients regardless of their care coordination needs.

Of note, 50% of the patients who did not have an appointment scheduled had a co-occurring substance use disorder (as compared to 37% of those who had an appointment scheduled). This group very likely included patients with substance use disorders that directly impacted their participation in care and follow-up, for example through refusal to accept substance use aftercare recommendations or a desire to leave the hospital prematurely when experiencing withdrawal symptoms or severe cravings. This finding could also reflect negative staff attitudes toward patients with substance use disorders that in turn can influence the quality of care.46 Co-occurring substance use is a well-established predictor of poor outcomes in psychiatric treatment, and hospital providers should be encouraged to pay greater attention to its management in acute settings. This is especially important given our finding that scheduling an aftercare appointment remained significantly associated with attending a visit following discharge within this subgroup of discharged patients with a co-occurring substance use disorders.

We also note geographic variation in the proportion of discharged patients for whom providers scheduled outpatient appointments. Among patients treated in hospitals in the large, urban/metropolitan area of New York City and the neighboring Hudson River region, 72% and 74%, respectively, had appointments scheduled as part of their discharge plans. These rates were lower than those observed among patients treated in hospitals in the Long Island (85%), Central (85%), and Western (89%) regions. This difference could reflect different standards of care across the regions or variability in data collection and reporting across the MBHOs. It may also reflect racial/ethnic disparities in treatment planning given the higher rates of minority populations treated in large urban/metropolitan areas such as greater New York City. We did not find an impact of race/ethnicity in our aforementioned statewide analysis, but given known racial/ethnic disparities in mental health treatment outcomes, this is an area deserving further focus.

Having an outpatient appointment scheduled as part of discharge planning increased the odds of attending a mental health service following discharge in all propensity score strata, indicating that this routine discharge planning activity was effective even in subgroups of patients at highest risk for not having an outpatient appointment scheduled. It would be neither feasible nor appropriate to attempt a randomized clinical trial to estimate the impact of routine discharge planning practice such as scheduling follow-up appointments. Instead, propensity score stratification allowed us to minimize potential bias associated with a range of patient, hospital, and service system covariates. Of note, these covariates included homelessness, the presence of a co-occurring substance use disorder, and a history of not engaging in care prior to admission, which are known predictors of failed care transitions and other adverse outcomes following discharge. Psychiatric inpatient clinicians may believe that routine discharge planning will be ineffective based upon their prior experiences with these complex patients, who commonly need more intensive care management interventions to increase the likelihood of successful care transitions.1,2,5-9 However, data from this study suggest that even among patients who were both least likely to receive discharge planning and most likely to have failed care transitions, scheduling an appointment with an outpatient mental health provider following discharge increased the odds of a successful care transition. Scheduling follow-up outpatient mental health appointments is a low-resource activity that should be considered an important target for hospital continuous quality improvement initiatives to improve discharge planning and increase rates of successful care transitions.

Several potential limitations of the study must be considered. Although we controlled for a range of patient, hospital, and service system variables, there are quite likely additional unmeasured variables that may affect attendance at outpatient appointments, eg, transportation limitations and attitudinal factors. There is also significant potential for measurement error given that we relied on multiple MBHOs independently reporting provider discharge planning activities. Findings from a Medicaid population may not generalize to commercial or Medicare populations due to a number of factors, and the New York State Medicaid population that served as our source for the study sample is likely to be different from other state Medicaid populations given variations in eligibility and enrollment practices across states.

Despite these potential limitations, the findings should be of interest to policy makers and clinical leaders. Future research should examine mechanisms underlying successful discharge planning and care transitions, including potentially relevant issues such as the role of familiarity of the community-based provider and whether additional discharge planning practices (forwarding care summaries, follow-up communications) further improve engagement in care following hospital discharge. An increasingly common approach to discharge planning for high-need patients is to refer to walk-in clinics, presuming that allowing the patient flexibility in determining when to follow-up after discharge may increase the likelihood of a successful transition. It is not clear whether failing to schedule appointments at fixed dates and times and allowing this flexibility will increase or decrease the likelihood of successful care transitions for various patient groups. Finally, for some patients, routine discharge planning practices such as scheduling outpatient mental health appointments will be insufficient to ensure a successful care transition. Research is needed to identify these individuals and determine what levels of increasingly intensive care transition interventions are necessary to meet their needs. In the meantime, findings underscore the importance of scheduling outpatient appointments as part of routine discharge planning for patients being treated in psychiatric inpatient units.

Submitted: March 14, 2020; accepted June 26, 2020.

Published online: September 15, 2020.

Disclosure of off-label usage: The authors have determined that, to the best of their knowledge, no investigational information about pharmaceutical agents or device therapies that is outside US Food and Drug Administration-approved labeling has been presented in this activity.

Financial disclosure: Drs Smith, Wall, Essock, Frimpong, Goldman, Radigan, Schneider, Dixon, and Olfson; Mss Haselden, Tang, and Wang; and Messrs Corbeil and Mascayano have no personal affiliations or financial relationships with any commercial interest to disclose relative to the article.

Funding/support: Research supported by a grant (R01MH106558) from the National Institute of Mental Health (NIMH).

Role of the sponsor: NIMH had no role in the conduct and publication of the study.

Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

REFERENCES

1.Stein BD, Kogan JN, Sorbero MJ, et al. Predictors of timely follow-up care among medicaid-enrolled adults after psychiatric hospitalization. Psychiatr Serv. 2007;58(12):1563-1569. PubMed CrossRef

2.Olfson M, Marcus SC, Doshi JA. Continuity of care after inpatient discharge of patients with schizophrenia in the Medicaid program: a retrospective longitudinal cohort analysis. J Clin Psychiatry. 2010;71(7):831-838. PubMed CrossRef

3.Compton MT, Craw J, Rudisch BE. Determinants of inpatient psychiatric length of stay in an urban county hospital. Psychiatr Q. 2006;77(2):173-188. PubMed CrossRef

4.Storm M, Husebø AML, Thomas EC, et al. Coordinating mental health services for people with serious mental illness: a scoping review of transitions from psychiatric hospital to community. Adm Policy Ment Health. 2019;46(3):352-367. PubMed CrossRef

5.Cuffel BJ, Held M, Goldman W. Predictive models and the effectiveness of strategies for improving outpatient follow-up under managed care. Psychiatr Serv. 2002;53(11):1438-1443. PubMed CrossRef

6.Nelson EA, Maruish ME, Axler JL. Effects of discharge planning and compliance with outpatient appointments on readmission rates. Psychiatr Serv. 2000;51(7):885-889. PubMed CrossRef

7.Carlisle CE, Mamdani M, Schachar R, et al. Aftercare, emergency department visits, and readmission in adolescents. J Am Acad Child Adolesc Psychiatry. 2012;51(3):283-293.e4. PubMed CrossRef

8.Bodén R, Brandt L, Kieler H, et al. Early non-adherence to medication and other risk factors for rehospitalization in schizophrenia and schizoaffective disorder. Schizophr Res. 2011;133(1-3):36-41. PubMed CrossRef

9.Mark TL, Tomic KS, Kowlessar N, et al. Hospital readmission among medicaid patients with an index hospitalization for mental and/or substance use disorder. J Behav Health Serv Res. 2013;40(2):207-221. PubMed CrossRef

10.Lin HC, Lee HC. The association between timely outpatient visits and the likelihood of rehospitalization for schizophrenia patients. Am J Orthopsychiatry. 2008;78(4):494-497. PubMed CrossRef

11.Valenstein M, Copeland LA, Blow FC, et al. Pharmacy data identify poorly adherent patients with schizophrenia at increased risk for admission. Med Care. 2002;40(8):630-639. PubMed CrossRef

12.Olfson M, Mechanic D, Hansell S, et al. Prediction of homelessness within three months of discharge among inpatients with schizophrenia. Psychiatr Serv. 1999;50(5):667-673. PubMed CrossRef

13.Herman DB, Susser ES, Jandorf L, et al. Homelessness among individuals with psychotic disorders hospitalized for the first time: findings from the Suffolk County Mental Health Project. Am J Psychiatry. 1998;155(1):109-113. PubMed CrossRef

14.Elbogen EB, Van Dorn RA, Swanson JW, et al. Treatment engagement and violence risk in mental disorders. Br J Psychiatry. 2006;189(4):354-360. PubMed CrossRef

15.Monahan J. The MacArthur studies of violence risk. Crim Behav Ment Health. 2002;12:S67-S72. CrossRef

16.Van Dorn RA, Desmarais SL, Petrila J, et al. Effects of outpatient treatment on risk of arrest of adults with serious mental illness and associated costs. Psychiatr Serv. 2013;64(9):856-862. PubMed CrossRef

17.Oliver P, Keen J, Rowse G, et al. The effect of time spent in treatment and dropout status on rates of convictions, cautions and imprisonment over 5 years in a primary care-led methadone maintenance service. Addiction. 2010;105(4):732-739. PubMed CrossRef

18.Katz IR, Peltzman T, Jedele JM, et al. Critical periods for increased mortality after discharge from inpatient mental health units: opportunities for prevention. Psychiatr Serv. 2019;70(6):450-456. PubMed CrossRef

19.Valenstein M, Kim HM, Ganoczy D, et al. Higher-risk periods for suicide among VA patients receiving depression treatment: prioritizing suicide prevention efforts. J Affect Disord. 2009;112(1-3):50-58. PubMed CrossRef

20.Vasiliadis HM, Ngamini-Ngui A, Lesage A. Factors associated with suicide in the month following contact with different types of health services in Quebec. Psychiatr Serv. 2015;66(2):121-126. PubMed CrossRef

21.Smith TE, Abraham M, Bolotnikova NV, et al. Psychiatric inpatient discharge planning practices and attendance at aftercare appointments. Psychiatr Serv. 2017;68(1):92-95. PubMed CrossRef

22.Fontanella CA, Hiance-Steelesmith DL, Bridge JA, et al. Factors associated with timely follow-up care after psychiatric hospitalization for youths with mood disorders. Psychiatr Serv. 2016;67(3):324-331. PubMed CrossRef

23.Cheng C, Chan CWT, Gula CA, et al. Effects of outpatient aftercare on psychiatric rehospitalization among children and emerging adults in Alberta, Canada. Psychiatr Serv. 2017;68(7):696-703. PubMed CrossRef

24.Carlisle CE, Mamdani M, Schachar R, et al. Predictors of psychiatric aftercare among formerly hospitalized adolescents. Can J Psychiatry. 2012;57(11):666-676. PubMed CrossRef

25.Bodenheimer T. Coordinating care—a perilous journey through the health care system. N Engl J Med. 2008;358(10):1064-1071. PubMed CrossRef

26.Kohn LT, Corrigan JM, Donaldson MS, eds. To Err is Human: Building a Safer Health System. Washington, DC: Institute of Medicine, National Academies Press; 1999.

27.National Committee for Quality Assurance, Healthcare Effectiveness Data and Information Set (HEDIS). NCQA website. http://www.ncqa.org/tabid/59/default.aspx. Accessed July 18, 2012.

28.Specifications Manual for Joint Commission National Quality Core Measures. The Joint Commission website. https://manual.jointcommission.org/releases/archive/TJC2010A/. Accessed January 14, 2020.

29.Rochester-Eyeguokan CD, Pincus KJ, Patel RS, et al. The current landscape of transitions of care practice models: a scoping review. Pharmacotherapy. 2016;36(1):117-133. PubMed CrossRef

30.Oduyebo I, Lehmann CU, Pollack CE, et al. Association of self-reported hospital discharge handoffs with 30-day readmissions. JAMA Intern Med. 2013;173(8):624-629. PubMed CrossRef

31.Schoen C, Osborn R, Huynh PT, et al. Taking the pulse of health care systems: experiences of patients with health problems in six countries. Health Aff (Millwood). 2005;24(suppl 1):W5-509-25. PubMed CrossRef

32.Cullen AE, Bowers L, Khondoker M, et al. Factors associated with use of psychiatric intensive care and seclusion in adult inpatient mental health services. Epidemiol Psychiatr Sci. 2018;27(1):51-61. PubMed CrossRef

33.Humensky JL, Fattal O, Feit R, et al. Factors associated with outpatient visit attendance after discharge from inpatient psychiatric units in a New York City hospital. Psychiatr Serv. 2017;68(6):624-627. PubMed CrossRef

34.Beadles CA, Ellis AR, Lichstein JC, et al. First outpatient follow-up after psychiatric hospitalization: does one size fit all? Psychiatr Serv. 2015;66(4):364-372. PubMed CrossRef

35.Batscha C, McDevitt J, Weiden P, et al. The effect of an inpatient transition intervention on attendance at the first appointment postdischarge from a psychiatric hospitalization. J Am Psychiatr Nurses Assoc. 2011;17(5):330-338. PubMed CrossRef

36.Pfeiffer PN, Ganoczy D, Zivin K, et al. Outpatient follow-up after psychiatric hospitalization for depression and later readmission and treatment adequacy. Psychiatr Serv. 2012;63(12):1239-1242. PubMed CrossRef

37.Steffen S, Kösters M, Becker T, et al. Discharge planning in mental health care: a systematic review of the recent literature. Acta Psychiatr Scand. 2009;120(1):1-9. PubMed CrossRef

38.Puschner B, Steffen S, Völker KA, et al. Needs-oriented discharge planning for high utilisers of psychiatric services: multicentre randomised controlled trial. Epidemiol Psychiatr Sci. 2011;20(2):181-192. PubMed CrossRef

39.Vigod SN, Kurdyak PA, Dennis CL, et al. Transitional interventions to reduce early psychiatric readmissions in adults: systematic review. Br J Psychiatry. 2013;202(3):187-194. PubMed CrossRef

40.Rennke S, Ranji SR. Transitional care strategies from hospital to home: a review for the neurohospitalist. Neurohospitalist. 2015;5(1):35-42. PubMed CrossRef

41.Priester MA, Browne T, Iachini A, et al. Treatment access barriers and disparities among individuals with co-occurring mental health and substance use disorders: an integrative literature review. J Subst Abuse Treat. 2016;61:47-59. PubMed CrossRef

42.Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005;43(11):1130-1139. PubMed CrossRef

43.van Walraven C, Austin PC, Jennings A, et al. A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data. Med Care. 2009;47(6):626-633. PubMed CrossRef

44Concept: Elixhauser Comorbidity Index. University of Manitoba, Rady Faculty of Health Sciences website. http://mchp-appserv.cpe.umanitoba.ca/viewConcept.php?printer=Y&conceptID=1436. Accesed Nov 21, 2019.

45.Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat Med. 2009;28(25):3083-3107. PubMed CrossRef

46.Brener L, von Hippel W, von Hippel C, et al. Perceptions of discriminatory treatment by staff as predictors of drug treatment completion: utility of a mixed methods approach. Drug Alcohol Rev. 2010;29(5):491-497. PubMed CrossRef