The Use of Technology in the Clinical Care of Depression: An Evidence Map

Related Blog

This work may not be copied, distributed, displayed, published, reproduced, transmitted, modified, posted, sold, licensed, or used for commercial purposes. By downloading this file, you are agreeing to the publisher’s Terms & Conditions.

The Use of Technology in the Clinical Care of Depression:

An Evidence Map

Eric A. Apaydin, PhDa,*; Alicia Ruelaz Maher, MDb; Laura Raaen, MPHa; Aneesa Motala, BAa; Sangita Baxi, MASa; Roberta M. Shanman, MSa; and Susanne Hempel, PhDa

Depression is a highly prevalent clinical condition. According to the US National Institute of Mental Health, the 12-month prevalence for major depressive disorder is 7%.1,2 Not only can it be emotionally challenging to be affected by these conditions, but the symptoms can lead to lost productivity, interpersonal difficulties, and even disability. Major depressive disorder is a leading cause of disability in the United States, second only to low back pain.3 Given the toll this illness can take on patients, providing adequate clinical care is critical.

Studies have found that at times, only 51% to 71% of those with major depression received treatment.4-6 Undertreatment is worse in the international context: only 16.5% of those with major depression in a group of high and middle income countries received minimally adequate treatment during a 1-year period.7 Traditionally, depression treatment has been delivered via face-to-face psychosocial interventions. However, cited barriers to this type of intervention include geographical factors, cost of the interventions, cost of transportation, and stigmatization of visiting a therapist.8,9 Recent technological advances in telephone and computer communication abilities have made it possible to circumvent some of these obstacles by providing care that is more private and convenient to the patient.10

Technology used in care can be any use of technology to expand access, exchange information between provider and patient, and deliver care in formats alternate to, or in addition to, face-to-face traditional treatment. The Health Resources and Services Administration defines telehealth11 as "the use of telecommunications and information technologies to share information and provide clinical care, education, public health, and administrative services at a distance." Telemedicine is using technology to improve a patient’s health by permitting 2-way, real time interactive communication between the patient and the physician or other health care provider. Many different types of technology can be used to support clinical interventions, including video, computer, telephone, smartphones, and others. The types of treatment can be provider-guided, self-guided, or a mixture of the two. The interventions can range from merely providing feedback or reminders to engaging in a complete course of psychotherapy. There are many online interventions available, including social support forums, psychoeducation, and self-help materials.12 In addition, psychotherapy can be delivered online. For instance, cognitive-behavioral therapy (CBT), which seeks to resolve problems by changing maladaptive thinking and behavior,13 can be delivered entirely or partially online. Partially online CBT can be guided in-person by a therapist using online materials. A systematic review of online-supported CBT found that it is both accepted by patients and effective for treating anxiety and depression.14 A recent evidence map by the Agency for Healthcare Research and Quality (AHRQ) that looked at systematic reviews of patient outcomes from telehealth for a range of medical conditions found that there is sufficient evidence to support the effectiveness of telehealth for psychotherapy as part of behavioral health.15

The known range and uses of the technologies require the assessments to cover an expansive body of literature. The breadth of this topic suggests that an evidence map is an effective way to summarize the existing research. Evidence maps synthesize large areas of research in an accessible and user-friendly manner. Through the use of visual displays of the volume and content areas of research, evidence maps can organize and help with understanding the evidence.16 While evidence maps involve methods similar to a traditional systematic review, uniform conduct and reporting standards for evidence maps do not currently exist.16 However, evidence maps can help to identify critical gaps in research. The purpose of this evidence map is to provide a broad overview of the research field of the use of technology in the clinical care in depression. This overview can assist practitioners, policy makers, and researchers who seek to incorporate technology into their clinical care or to identify research gaps in the use of such technology.

Previous reviews and meta-analyses of technological interventions for depression have been conducted,17-19 but they were relatively small,17-19 focused heavily on online CBT,17 excluded US studies,19 included non-RCTs,17,19 or included studies of patients with non-depression comorbidities.17-19 This evidence map provides a visual overview of the distribution of evidence for the different types of technology for which there are randomized controlled trials (RCTs) examining their use in the treatment of depression disorders. The map concentrates on evaluations using this robust study design that allows drawing strong conclusions about the effectiveness of the technology. Furthermore, the evidence map focuses on treatment-related technology that has the potential for use in clinical depression care. Although technological treatment such as biofeedback, transcranial magnetic stimulation, or electroconvulsive therapy may be used in depression, this evidence map is limited to telehealth, telemedicine, and online interventions. While some of the included interventions could not exist without technology (eg, auto-reminders), most are digital versions of existing face-to-face treatments (eg, provider therapy). The map documents both what is known and where there is little or no evidence, and it describes the volume, nature, and characteristics of research in this field.

Key Questions

The following areas are addressed in this evidence map:

1. How many studies evaluate the use of technology in depression care? What are the types and functions of the evaluated technologies?
2. What is the direction of effect in studies evaluating the technology?
3. How does the effectiveness of the evaluated technologies change over time, by technology type and function?

METHODS

The evidence map is based on a systematic review that is registered in PROSPERO (registration number CRD42017069691), an international prospective registry for systematic reviews. The evidence map also complies with criteria set forth for broad overviews such as umbrella reviews.20

Sources

We searched the electronic databases PubMed, PsycINFO, and Web of Science to identify published English-language RCTs. We reference-mined published reviews by screening studies included in pertinent reviews on the topic. In addition, we consulted with topic experts reviewing the draft report of this evidence map to check the sample for completeness.

Search Strategy

The search strategy was developed by an evidence-based practice center librarian together with a psychiatrist familiar with the clinical conditions and treatment approaches, informed by search results of the prior feasibility scans conducted for this project and existing reviews on telehealth. The search was done as part of a larger review of both depression and anxiety disorders, with those studies specific to depression separated out for the focus of this current evidence map. The search strategy is documented in Appendix 1.

We retrieved and screened full texts of RCTs evaluating any type of technology in the clinical care in depression to determine whether the publication meets the inclusion criteria. The results of the literature search and screening are documented in a PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) literature flow diagram (Figure 1).

Inclusion Screening

Two reviewers (the project lead, who is an experienced reviewer and psychiatrist, and a RAND research associate with extensive literature review experience) independently screened titles and abstracts of retrieved citations. Citations judged as potentially eligible by 1 or both reviewers were obtained as full text.

Following a pilot session to ensure similar interpretation of the inclusion and exclusion criteria, the 2 reviewers independently screened full text publications against the inclusion and exclusion criteria. Any disagreements were resolved through discussion in the review team. Reasons for exclusion are recorded in an electronic database.

Eligibility Criteria

Study inclusion and exclusion criteria can be summarized in the following "PICOTSS" framework (participants, interventions, comparators, outcomes, timing, setting, and study design).

• Participants.

  Inclusion criteria. Studies of male and female participants, 18 years of age or older, were eligible for inclusion. Children were excluded to limit the size of the included evidence base. Participants could have any type of depression symptoms, diagnosis, or disorders, including major depressive disorder, dysthymia, premenstrual dysphoric disorder, or other specified or unspecified depressive disorder. Participants with depression due to a substance/medication or medical condition were excluded.

  Exclusion criteria. Participants with other diagnoses such as obsessive-compulsive disorder or posttraumatic stress disorder not classified as depression disorders according to the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) were excluded. Studies exclusively reporting on participants with self-limited, situational depression were excluded. Studies that did not include a majority of patients with depression disorders were excluded. Studies that involved a population of patients with a particular medical diagnosis or situation (such as caregivers), in which the intervention addressed primarily the medical or situational needs and secondarily depression symptoms, were excluded.

• Interventions. Studies evaluating technology to directly treat depression or in the clinical care in depressive disorders were eligible. Interventions focused on collaborative care, such as between a primary care provider and a psychiatrist, were excluded. Interventions designed to treat or manage another condition, where the effects of the intervention on depression were secondary, were excluded. We included phone, smartphone, video, or computer-based technology for clinical care, but technological treatment such as biofeedback, transcranial magnetic stimulation, or electroconvulsive therapy offered in a health care setting was excluded. The interventions could be delivered in addition to nonpharmacologic or medication-based care.
• Comparators. Studies that compare to depression disorder treatment, with or without the use of technology, wait-list control, or no treatment were eligible.
• Outcomes. Studies reporting symptom improvement, response to treatment, and remission of depression were eligible.
• Timing. Studies could involve any treatment duration and any follow-up period.
• Setting. Studies were not limited by setting.
• Study design. Studies that did not randomize to intervention versus comparator were excluded. Studies using any form of randomization (eg, parallel and crossover) were acceptable and included.

Only studies published in English were eligible for inclusion in this evidence map.

Data Extraction

An evidence map aims to provide a broad overview of a large literature base. To ensure the overview is meaningful, the number of variables that can be abstracted and displayed in result figures must be selected carefully. Data collection forms were designed by the project team in advance of literature screening. Reviewers pilot tested the data collection forms on 10 randomly selected studies to ensure agreement of interpretation. The reviewers abstracted study-level data using database software designed for systematic reviews. Unclear descriptions and ambiguities were discussed in the review team.

A listing of data abstraction variables follows:

• Study size. Number of randomized participants (if this was not available, the number of participants analyzed).
• Type of technology. Categories of the type of technology used in supporting clinical care, according to the type of technology most likely chosen by the participant, based on the lowest level of technology most likely to fully implement the intervention: phone only, computer only (online treatment, e-mail communication with providers), smartphone only (app or texting), videoconferencing (interaction with provider, needs a dedicated piece of technology with a camera, not video or webcam chat over a computer), multiple technology (explicitly using more than 1 technology, not for studies in which patients have the choice between using a normal phone or a smartphone), and other (technology could not be classified into the preceding categories).
• Function of technology. Categories of the function of the technology in clinical care: automated reminders (fully automated), provider-sent reminders (requires interaction with provider, provider-initiated), patients’ self-directed support (online resources for patients, peer support), provider support (eg, provider-run support group), self-directed psychotherapy (eg, online modules), provider-directed psychotherapy (virtual interaction with provider, eg, interaction by phone or online), automated feedback (eg, after task completion; patient has instructions to record task and to watch it back with specific instruction), provider feedback (eg, provider watches recorded video from patient and gives feedback), multiple functions, and other (function could not be classified into the preceding categories).
• Effectiveness signal. Category of directionality of effectiveness measure (ie, are the symptoms of depression improved relative to no technology?): negative (ie, intervention group outcomes are worse than the control group or baseline outcomes), unclear (effect unclear, no change, or not applicable because 2 active technologies were compared), or positive (ie, intervention group outcomes are better than control group or baseline outcomes). The category is based on results as reported by the original study authors and will be based on the outcomes most relevant to the diagnosis in the respective study (eg, depression symptoms in patients treated for depression) compared to no use of technology.
• Publication year. Year the study was published.

Quality of Evidence

This evidence map quantifies the number of RCTs available for the areas of interest. The map documents technologies in clinical care of depression that have been evaluated with the strong study design of RCT. RCTs allow confident evidence statements, and available research will be of particular interest to policy makers. The evidence map further broadly characterizes the quality of evidence using 2 aspects of the GRADE approach21: imprecision and inconsistency.

For documented evidence characteristics, the evidence map figures display the size of the study. The size of the study is instrumental in determining for which interventions precise estimates are currently available in the literature. Given the large number of pertinent studies, this aspect evaluates the potential for precise estimates, and the study size is used as a proxy.

Furthermore, we highlight intervention evaluations and findings that have been consistently replicated in multiple RCTs published by independent researcher groups.

RESULTS

The search and selection of RCTs are summarized in the literature flow diagram in Figure 1. Database searches of published literature and reference mining of accepted RCTs resulted in 4,062 potentially relevant articles. After dual review of titles and abstracts, 1,366 RCTs were selected for full-text dual review. Of these, 161 RCTs met the inclusion criteria and are included in the evidence map.22-182

The earliest study was published in 1991, and the number of studies published each year increased from 1 in that year to 22 in 2016 (9 were published in 2017 when we conducted our search). The evidence table (Table 1) lists the details of the included studies.

Key Question 1: How Many Studies Have Evaluated the Use of Technology in Depression Care? What Are the Types and Function of the Evaluated Technologies?

To answer the key questions, we have documented the evidence in scatter plots. Each point in the plots represents 1 of the 161 included RCTs.

Figure 2 shows the types of technology and functions of the technology used in treatment for patients with depression, as well as the size of each study. In examining this evidence map, the key questions regarding technology type and functions are answered. The majority of the studies utilized computer (98 studies; 61%) and phone (37 studies; 23%) technology, followed by 12 studies (7%) using more than 1 technology. Eight studies used smartphones (5%), and 4 used videoconferences (2%). The 2 studies in the "other" category (1%) included a computer-telephone system and an analog videophone.

Most studies (51; 32%) examined technologies with multiple functions, patient self-directed therapy (41; 25%), and provider-directed therapy (38; 24%). The category "multiple types of technology" often indicated a program designed to be used at home on the computer, with additional phone or text support. For example, the participants may complete self-directed psychotherapy via computer with e-mail and SMS reminders, requiring that they also use a smartphone to receive the SMS texts.183 Often, this was completing self-directed psychotherapy with additional support from a therapist by e-mail and phone calls.127 A minority of studies looked at provider-directed support (11; 7%), patient self-directed support (7; 4%), provider-directed feedback (1; 1%), and auto-reminders (1; 1%). There were 11 studies (7%) of "other" technological functions. The most common type/function combinations were computer/patient self-therapy (35 studies, 22% of total), computer/multiple interventions (34 studies, 21%), and phone/provider-directed therapy (18 studies, 11%).

In the studies examined, 96% (155 out of 161 studies) included more than 31 participants and 63% (101) included 100 or more participants. There were 54 studies (34%) that included between 31 and 99 participants and 6 small studies (4%) that included up to 30 participants. The smallest study included 7 participants.184 The largest study, with 2,794 participants, recruited participants who spontaneously accessed the MoodGym website.49

Key Question 2: What Is the Direction of Effect in Studies Evaluating the Technology?

We also reviewed the reported treatment effects for a broad overview of the effectiveness of the technology. Figure 3 gives further information about the evidence base of technology utilized to treat depression by documenting the direction of effect of the study.

Regardless of the function of the technology used, 90% (145) of studies reported that the intervention had a positive outcome on symptoms, particularly for depression. There were 15 studies (9%) that reported no change, particularly for self-directed interventions, such as self-directed psychotherapy (8 studies) and patient self-directed support (2 studies), but also provider-directed psychotherapy (1 study), provider support (2 studies), and multiple (2 studies). One study of provider-directed therapy had outcomes that were judged to be unclear (1%).

Key Question 3: How Does Effectiveness of the Evaluated Technologies Change Over Time, by Technology Type and Function?

Technology changes over time, and newer technology is likely to be more highly evaluated. To disentangle the volume of evaluations and the recency of technology, we also examined treatment effects by publication year. Figures 4 and 5 detail reported treatment effectiveness by publication year and technology type and function, respectively.

Most evaluations of computer (62 studies; 63%), smartphone (7 studies; 88%), and multiple interventions (10 studies; 83%) were published within the last 5 years of this search. Evaluations of videoconference (2 studies within the last 5 years; 50%) and other interventions (1 study within the last 5 years; 50%) were published evenly before and after this period. A minority of studies of phone interventions (12 studies, 32%) were published after 2012. A slight majority of all studies were published in the last 5 years of this search (94 studies; 58%) regardless of intervention technology. Eighty-eight of these studies (94%) reported positive effects of the studied technology. Conversely, only 57 of the 67 studies (85%) published in or before 2012 reported positive effects.

Most of the included studies of self-therapy (27 studies; 66%), provider therapy (20 studies; 53%), multiple functions (33 studies; 65%), and other functions (7 studies; 64%) were published after 2012. Fewer than half of studies of interventions involving self-support (3 studies; 43%) and provider support (2 studies; 18%) were published in the last 5 years of this search. Only 1 study of auto-reminders (published in 2016) and 1 study of provider feedback (published in 2015) met our inclusion criteria.

DISCUSSION

This evidence map provides an overview of the broad research field of the use of technology in the clinical care in depression. We summarized the evidence published in RCTs.

We identified 161 published RCTs that included different types of technologies that performed different functions in the clinical care of depressive disorders. We chose the type of technologies that scoping searches revealed were most common, including those requiring the use of computer, phone, smartphone, videoconference, or multiple technologies, while allowing for uncommon technologies in the category labeled other. In addition, we further defined the treatments by identifying what function the technology served, including auto-reminders, support given between patients, patient self-directed psychotherapy, provider support, provider feedback, provider-directed psychotherapy, multiple functions within 1 treatment, and other. The most common type of evaluated technology application is currently computer-based psychotherapy. This technology has the potential to expand access to mental health care.

A key factor in deciding whether to implement new technologies is whether or not they are effective in reducing symptoms. One hundred forty-five of the 161 studies found a positive outcome in depression symptoms at follow-up assessment compared to baseline. However, 15 found no change, and 1 reported unclear results, and these were primarily self-directed interventions such as self-directed psychotherapy or patient directed support.

Technology changes over time, and so it is expected that newer technologies were evaluated more recently. Most of the included studies involving a computer, smartphone, or multiple technologies were published in the last 5 years of this search, while most evaluations of phone interventions were published before 2013. It is less clear that the evaluation of technology function changed over time. A majority of included studies of self-therapy, provider therapy, multiple functions, and other functions were published in the last 5 years of the search, but so were a majority of all included studies. This increase in publication of studies of these technology functions may just represent an overall increase in the evaluation of technological interventions for depression.

Limitations

The evidence map is well suited to identify promising areas that should be reviewed in more detail in subsequent systematic reviews. However, an evidence map is a broad overview that cannot provide accurate estimates of the effectiveness of interventions or the magnitude of effects across studies.

Given the large body of literature, evidence maps must focus on a limited number of characteristics. We attempted to capture the types of technology as well as the functions of technology that a scoping search suggested would be most likely. However, there remain a large number of studies that fell into the categories "other" or "multiple." These studies addressed unique and study-specific technology. Studies could also use multiple types of technology and multiple functions within the technology type. Almost all of these studies of combined technologies and functions showed positive symptom outcomes, and therefore there may be technologies that are more effective when combined, though that is not defined in this evidence map. In addition, it is possible that by combining technologies or functions, one may have a more effective treatment, but the particular items combined would need to be delineated more than an evidence map is able to do.

It is also noteworthy that in this broad-stroke overview, the large majority of studies appeared to have positive effects. When abstracting data regarding outcomes, we allowed for any measurement of symptom improvement. We abstracted data provided by the authors regarding whether the treatment had a positive outcome on symptoms. This outcome could be determined in a variety of ways including patient report, clinician assessment, or need for further care. Symptoms were assessed by a variety of scales or without a scale. It is likely that there would have been a different number of studies with a positive outcome if all studies were held to the same standard for determining this result. A meta-analysis is needed to determine the size of the treatment effect and for a more refined discussion of the effects of technology in depression treatment.

The vast majority of all studies (90%) reported positive effects, but this proportion was higher (94%) among studies published in the last 5 years of the search than among those published before (85%). This discrepancy could represent publication bias,185 in that more studies reporting positive effects may have been published after the effects of such interventions became well known. We did not conduct a meta-analysis, and so we cannot employ traditional statistical methods186 to detect and correct for publication bias. However, such a bias is likely irrelevant, as the most commonly included interventions, self- and provider-directed therapy, are already well established in their effectiveness.187,188

Comparison to Other Research

Our review is consistent with other research in this area. A systematic review and meta-analysis of computer based psychological treatments for depression analyzed 40 studies and found this treatment to be effective.17 A meta-analysis of computerized cognitive training for depression examined 9 trials and found this treatment to be effective, as well.189 A meta-analysis of computerized treatment for depression also found positive effects over 12 trials,18 and a narrative review of 13 online self-help programs for mental illness found the interventions to be feasible in low and middle income countries.19

A recent evidence map created by AHRQ provided an overview of the evidence about telehealth, in general, which included behavioral health.15 The report used a bubble plot format to display the distribution of evidence from systematic reviews in terms of volume (number of reviews and number of patients in included studies), conclusions about benefit by clinical focus, and telehealth function. The authors also determined how much evidence was available about combinations of clinical areas and telehealth functions reported in the existing systematic reviews. For behavioral health, they utilized systematic reviews for the treatment of conditions including depression, anxiety, and addiction (including substance abuse, smoking cessation, alcohol abuse, and pathological gambling). The map included 7 systematic reviews, examining 137 studies (83 RCTs) and 32,770 patients. They determined that there was sufficient evidence to support the effectiveness of telehealth for psychotherapy as a part of behavioral health.15 Fifty-eight of their RCTs looked at psychotherapy, compared to 79 of ours. Also, we further divided by provider-directed and self-directed psychotherapy and separated these studies by type of technology. Most of the behavioral health studies they found reported benefit or potential benefit, except for 1 that found insufficient evidence for correctional or forensic psychiatry. This is similar to our findings of positive outcomes for the majority of our studies.

Another review summarized the findings of 10 systematic reviews of telepsychiatry. The emphasis of this review was on the feasibility of use in resource constrained environments. The studies in the reviews were conducted in the United States, Canada, Europe, Australia, Japan, and Hong Kong.190 Many of the studies presented in this evidence map were in resource-constrained environments, with the express purpose of finding more economical ways to bring mental health treatment to these areas or populations. The researchers reviewing the systematic reviews reported that the reviews were of acceptable quality but that common deficiencies were lack of grading the strength of evidence or linking the quality of the included studies to the conclusions. Consistent with our evidence map finding, the review of reviews concluded that telepsychiatry is as effective as face-to-face treatment and does improve symptoms.190

Future Research

This evidence map documents the existing literature as well as research gaps in the evidence base. In fact, an evidence map is often explicitly used not only to identify relevant evidence, but also to initiate the process of developing a research agenda that can address remaining gaps.16 The figures indicate where there are few or no published studies for some types of technology, like videoconference or smartphones, or for a particular function the technology serves, like provider feedback or auto-reminders. Research gaps were not apparent for studies that evaluate symptom outcomes, as all did so. However, it was interesting to note that most of the studies that did not find a positive change were those of self-directed psychotherapy or patient-directed support. Further research into whether self-directed versus provider-directed interventions are more effective would be valuable in determining whether the allocation of resources to provider-directed interventions may be warranted. Having a provider administer treatment can greatly affect cost and feasibility of implementation and therefore should be examined. Studies of smartphones were less numerous than studies of phone or computer. Given the fact that so many patients have smartphone technology available to them, more studies involving this technology may be beneficial.

CONCLUSION

This evidence map provides a broad overview about the types and functions of technology used in the clinical care of depressive disorders. The most research was available for computer and phone treatments for self- or provider-directed psychotherapy. The majority of studies reported symptom improvement and were published in the last 5 years of this search.

Several areas of future research were identified including research regarding smartphone capabilities, as these are becoming far more utilized by patients, yet studies utilizing smartphones are far fewer than phone and computer studies. Also, investigation into whether provider-directed versus self-directed interventions are more effective is needed, as this may affect resource allocation.

Limitations included a large number of studies whose technology types and/or functions did not fit into our predetermined categories. Delineation of these technologies, as well as further exploration into the studies that combined technology types or treatments, would give more extensive data.

Submitted: January 10, 2018; accepted June 26, 2018.

Published online: August 21, 2018.

Potential conflicts of interest: Drs Apaydin, Maher, and Hempel and Mss Raaen, Motala, Baxi, and Shanman have no conflicts of interest to disclose.

Funding/support: This research is sponsored by the US Department of Defense, Psychological Health Center of Excellence (PCoE), contract number HQ0034-16-D-0001.

Disclaimer: The findings and conclusions in this manuscript are those of the authors and do not necessarily represent the views of the funder.

Acknowledgments: The authors thank Thomas Concannon, PhD (RAND Corporation); Bradley Belsher, PhD (Psychological Health Center of Excellence, US Department of Defense); Rebecca Morgan, PhD (Case Western Reserve University); and Nigel Bush, PhD (Psychological Health Center of Excellence, US Department of Defense) for helpful comments and Patty Smith (RAND Corporation) for administrative support of this work. They also appreciate the contributions of Joyce Marks, BS (RAND Corporation); Christopher Maerzluft, BA (RAND Corporation); and Geoffrey Grimm, MA (RAND Corporation) in programming and developing the figures. The acknowledged individuals have no conflicts of interest to disclose.

Supplementary material: Available at PSYCHIATRIST.COM.

REFERENCES

100. Lappalainen P, Granlund A, Siltanen S, et al. ACT Internet-based vs face-to-face? a randomized controlled trial of two ways to deliver Acceptance and Commitment Therapy for depressive symptoms: an 18-month follow-up. Behav Res Ther. 2014;61:43-54. PubMed CrossRef

101. Lappalainen P, Langrial S, Oinas-Kukkonen H, et al. Web-based acceptance and commitment therapy for depressive symptoms with minimal support: a randomized controlled trial. Behav Modif. 2015;39(6):805-834. PubMed CrossRef

102. Lerner D, Adler DA, Rogers WH, et al. A randomized clinical trial of a telephone depression intervention to reduce employee presenteeism and absenteeism. Psychiatr Serv. 2015;66(6):570-577. PubMed CrossRef

103. Littlewood E, Duarte A, Hewitt C, et al; REEACT Team. A randomised controlled trial of computerised cognitive behaviour therapy for the treatment of depression in primary care: the Randomised Evaluation of the Effectiveness and Acceptability of Computerised Therapy (REEACT) trial. Health Technol Assess. 2015;19(101):viii, xxi-171. PubMed CrossRef

104. Lokman S, Leone SS, Sommers-Spijkerman M, et al. Complaint-directed mini-interventions for depressive complaints: a randomized controlled trial of unguided web-based self-help interventions. J Med Internet Res. 2017;19(1):e4. PubMed CrossRef

105. Ludman EJ, Simon GE, Tutty S, et al. A randomized trial of telephone psychotherapy and pharmacotherapy for depression: continuation and durability of effects. J Consult Clin Psychol. 2007;75(2):257-266. PubMed CrossRef

106. Luxton DD, Pruitt LD, Wagner A, et al. Home-based telebehavioral health for US military personnel and veterans with depression: a randomized controlled trial. J Consult Clin Psychol. 2016;84(11):923-934. PubMed CrossRef

107. Ly KH, Trüschel A, Jarl L, et al. Behavioural activation versus mindfulness-based guided self-help treatment administered through a smartphone application: a randomised controlled trial. BMJ Open. 2014;4(1):e003440. PubMed CrossRef

108. Ly KH, Topooco N, Cederlund H, et al. Smartphone-supported versus full behavioural activation for depression: a randomised controlled trial. PLoS One. 2015;10(5):e0126559. PubMed CrossRef

109. Lynch D, Tamburrino M, Nagel R, et al. Telephone-based treatment for family practice patients with mild depression. Psychol Rep. 2004;94(3 pt 1):785-792. PubMed CrossRef

110. Mackinnon A, Griffiths KM, Christensen H. Comparative randomised trial of online cognitive-behavioural therapy and an information website for depression: 12-month outcomes. Br J Psychiatry. 2008;192(2):130-134. PubMed CrossRef

111. Marasinghe RB, Edirippulige S, Kavanagh D, et al. Effect of mobile phone-based psychotherapy in suicide prevention: a randomized controlled trial in Sri Lanka. J Telemed Telecare. 2012;18(3):151-155. PubMed CrossRef

112. Meyer B, Berger T, Caspar F, et al. Effectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trial. J Med Internet Res. 2009;11(2):e15. PubMed CrossRef

113. Meyer B, Bierbrodt J, Schroder J, et al. Effects of an Internet intervention (Deprexis) on severe depression symptoms: randomized controlled trial. Internet Interventions. 2015;2(1):48-59. CrossRef

114. Migliorini C, Sinclair A, Brown D, et al. A randomised control trial of an Internet-based cognitive behaviour treatment for mood disorder in adults with chronic spinal cord injury. Spinal Cord. 2016;54(9):695-701. PubMed CrossRef

115. Milgrom J, Danaher BG, Gemmill AW, et al. Internet cognitive behavioral therapy for women with postnatal depression: a randomized controlled trial of MumMoodBooster. J Med Internet Res. 2016;18(3):e54. PubMed CrossRef

116. Millán-Calenti JC, Lorenzo T, Núñez-Naveira L, et al. Efficacy of a computerized cognitive training application on cognition and depressive symptomatology in a group of healthy older adults: a randomized controlled trial. Arch Gerontol Geriatr. 2015;61(3):337-343. PubMed CrossRef

117. Mohr DC, Hart SL, Julian L, et al. Telephone-administered psychotherapy for depression. Arch Gen Psychiatry. 2005;62(9):1007-1014. PubMed CrossRef

118. Mohr DC, Carmody T, Erickson L, et al. Telephone-administered cognitive behavioral therapy for veterans served by community-based outpatient clinics. J Consult Clin Psychol. 2011;79(2):261-265. PubMed CrossRef

119. Mohr DC, Ho J, Duffecy J, et al. Effect of telephone-administered vs face-to-face cognitive behavioral therapy on adherence to therapy and depression outcomes among primary care patients: a randomized trial. JAMA. 2012;307(21):2278-2285. PubMed CrossRef

120. Mohr DC, Duffecy J, Ho J, et al. A randomized controlled trial evaluating a manualized TeleCoaching protocol for improving adherence to a web-based intervention for the treatment of depression. PLoS One. 2013;8(8):e70086. PubMed CrossRef

121. Montero-Marí­n J, Araya R, Pérez-Yus MC, et al. An internet-based intervention for depression in primary care in Spain: a randomized controlled trial. J Med Internet Res. 2016;18(8):e231. PubMed CrossRef

122. Moreno FA, Chong J, Dumbauld J, et al. Use of standard Webcam and Internet equipment for telepsychiatry treatment of depression among underserved Hispanics. Psychiatr Serv. 2012;63(12):1213-1217. PubMed CrossRef

123. Morris RR, Schueller SM, Picard RW. Efficacy of a Web-based, crowdsourced peer-to-peer cognitive reappraisal platform for depression: randomized controlled trial. J Med Internet Res. 2015;17(3):e72. PubMed CrossRef

124. Moritz S, Schilling L, Hauschildt M, et al. A randomized controlled trial of internet-based therapy in depression. Behav Res Ther. 2012;50(7-8):513-521. PubMed CrossRef

125. Mota Pereira J. Facebook enhances antidepressant pharmacotherapy effects. Sci World J. 2014;2014:892048. PubMed CrossRef

126. Neugebauer R, Kline J, Markowitz JC, et al. Pilot randomized controlled trial of interpersonal counseling for subsyndromal depression following miscarriage. J Clin Psychiatry. 2006;67(8):1299-1304. PubMed CrossRef

127. Newby J, Robins L, Wilhelm K, et al. Web-based cognitive behavior therapy for depression in people with diabetes mellitus: a randomized controlled trial. J Med Internet Res. 2017;19(5):e157. PubMed CrossRef

128. Ngai FW, Wong PW, Leung KY, et al. The effect of telephone-based cognitive-behavioral therapy on postnatal depression: a randomized controlled trial. Psychother Psychosom. 2015;84(5):294-303. PubMed CrossRef

129. Nobis S, Lehr D, Ebert DD, et al. Efficacy of a web-based intervention with mobile phone support in treating depressive symptoms in adults with type 1 and type 2 diabetes: a randomized controlled trial. Diabetes Care. 2015;38(5):776-783. PubMed CrossRef

130. Nyström MBT, Stenling A, Sjöström E, et al. Behavioral activation versus physical activity via the internet: a randomized controlled trial. J Affect Disord. 2017;215:85-93. PubMed CrossRef

131. O’ Leary K, Dockray S. The effects of two novel gratitude and mindfulness interventions on well-being. J Altern Complement Med. 2015;21(4):243-245. PubMed CrossRef

132. O’ Mahen HA, Woodford J, McGinley J, et al. Internet-based behavioral activation—treatment for postnatal depression (Netmums): a randomized controlled trial. J Affect Disord. 2013;150(3):814-822. PubMed CrossRef

133. O’ Neil A, Taylor B, Hare DL, et al; MoodCare Investigator Team. Long-term efficacy of a tele-health intervention for acute coronary syndrome patients with depression: 12-month results of the MoodCare randomized controlled trial. Eur J Prev Cardiol. 2015;22(9):1111-1120. PubMed CrossRef

134. Perini S, Titov N, Andrews G. Clinician-assisted Internet-based treatment is effective for depression: randomized controlled trial. Aust N Z J Psychiatry. 2009;43(6):571-578. PubMed CrossRef

135. Pictet A, Jermann F, Ceschi G. When less could be more: Investigating the effects of a brief internet-based imagery cognitive bias modification intervention in depression. Behav Res Ther. 2016;84:45-51. PubMed CrossRef

136. Piette JD, Richardson C, Himle J, et al. A randomized trial of telephonic counseling plus walking for depressed diabetes patients. Med Care. 2011;49(7):641-648. PubMed CrossRef

137. Pinto MD, Hickman RL Jr, Clochesy J, et al. Avatar-based depression self-management technology: promising approach to improve depressive symptoms among young adults. Appl Nurs Res. 2013;26(1):45-48. PubMed CrossRef

138. Pinto MD, Greenblatt AM, Hickman RL, et al. Assessing the critical parameters of eSMART-MH: a promising avatar-based digital therapeutic intervention to reduce depressive Symptoms. Perspect Psychiatr Care. 2016;52(3):157-168. PubMed CrossRef

139. Pots WT, Fledderus M, Meulenbeek PA, et al. Acceptance and commitment therapy as a web-based intervention for depressive symptoms: randomised controlled trial. Br J Psychiatry. 2016;208(1):69-77. PubMed CrossRef

140. Proyer RT, Gander F, Wellenzohn S, et al. Positive psychology interventions in people aged 50-79 years: long-term effects of placebo-controlled online interventions on well-being and depression. Aging Ment Health. 2014;18(8):997-1005. PubMed CrossRef

141. Pugh NE, Hadjistavropoulos HD, Dirkse D. A randomised controlled trial of therapist-assisted, internet-delivered cognitive behavior therapy for women with maternal depression. PLoS One. 2016;11(3):e0149186. PubMed CrossRef

142. Pyne JM, Fortney JC, Tripathi SP, et al. Cost-effectiveness analysis of a rural telemedicine collaborative care intervention for depression. Arch Gen Psychiatry. 2010;67(8):812-821. PubMed CrossRef

143. Richards D, Timulak L, Hevey D. A comparison of two online cognitive-behavioural interventions for symptoms of depression in a student population: the role of therapist responsiveness. Couns Psychother Res. 2013;13(3):184-193. CrossRef

144. Richards D, Timulak L, O’ Brien E, et al. A randomized controlled trial of an internet-delivered treatment: its potential as a low-intensity community intervention for adults with symptoms of depression. Behav Res Ther. 2015;75:20-31. PubMed CrossRef

145. Rickhi B, Kania-Richmond A, Moritz S, et al. Evaluation of a spirituality informed e-mental health tool as an intervention for major depressive disorder in adolescents and young adults: a randomized controlled pilot trial. BMC Complement Altern Med. 2015;15(1):450. PubMed CrossRef

146. Rickles NM, Svarstad BL, Statz-Paynter JL, et al. Pharmacist telemonitoring of antidepressant use: effects on pharmacist-patient collaboration. J Am Pharm Assoc (2003). 2005;45(3):344-353. PubMed CrossRef

147. Roepke AM, Jaffee SR, Riffle OM, et al. Randomized controlled trial of superbetter, a smartphone-based/internet-based self-help tool to reduce depressive symptoms. Games Health J. 2015;4(3):235-246. PubMed CrossRef

148. Rollman BL, Belnap BH, LeMenager MS, et al. Telephone-delivered collaborative care for treating post-CABG depression: a randomized controlled trial. JAMA. 2009;302(19):2095-2103. PubMed CrossRef

149. Romero-Sanchiz P, Nogueira-Arjona R, Garcí­a-Ruiz A, et al. Economic evaluation of a guided and unguided internet-based CBT intervention for major depression: Results from a multi-center, three-armed randomized controlled trial conducted in primary care. PLoS One. 2017;12(2):e0172741. PubMed CrossRef

150. Rosso IM, Killgore WD, Olson EA, et al. Internet-based cognitive behavior therapy for major depressive disorder: a randomized controlled trial. Depress Anxiety. 2017;34(3):236-245. PubMed CrossRef

151. Ruskin PE, Silver-Aylaian M, Kling MA, et al. Treatment outcomes in depression: comparison of remote treatment through telepsychiatry to in-person treatment. Am J Psychiatry. 2004;161(8):1471-1476. PubMed CrossRef

152. Ruwaard J, Schrieken B, Schrijver M, et al. Standardized web-based cognitive behavioural therapy of mild to moderate depression: a randomized controlled trial with a long-term follow-up. Cogn Behav Ther. 2009;38(4):206-221. PubMed CrossRef

153. Salisbury C, O’ Cathain A, Edwards L, et al. Effectiveness of an integrated telehealth service for patients with depression: a pragmatic randomised controlled trial of a complex intervention. Lancet Psychiatry. 2016;3(6):515-525. PubMed CrossRef

154. Sandoval LR, Buckey JC, Ainslie R, et al. Randomized controlled trial of a computerized interactive media-based problem solving treatment for depression. Behav Ther. 2017;48(3):413-425. PubMed CrossRef

155. Selmi PM, Klein MH, Greist JH, et al. Computer-administered therapy for depression. MD Comput. 1991;8(2):98-102. PubMed

156. Shamshiri Milani H, Azargashb E, Beyraghi N, et al. Effect of telephone-based support on postpartum depression: a randomized controlled trial. Int J Fertil Steril. 2015;9(2):247-253. PubMed

157. Sheeber LB, Seeley JR, Feil EG, et al. Development and pilot evaluation of an Internet-facilitated cognitive-behavioral intervention for maternal depression. J Consult Clin Psychol. 2012;80(5):739-749. PubMed CrossRef

158. Simon GE, Ludman EJ, Tutty S, et al. Telephone psychotherapy and telephone care management for primary care patients starting antidepressant treatment: a randomized controlled trial. JAMA. 2004;292(8):935-942. PubMed CrossRef

159. Simon GE, Ludman EJ, Operskalski BH. Randomized trial of a telephone care management program for outpatients starting antidepressant treatment. Psychiatr Serv. 2006;57(10):1441-1445. PubMed CrossRef

160. Simon GE, Ralston JD, Savarino J, et al. Randomized trial of depression follow-up care by online messaging. J Gen Intern Med. 2011;26(7):698-704. PubMed CrossRef

161. Spek V, Nyklí­cek I, Smits N, et al. Internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years old: a randomized controlled clinical trial. Psychol Med. 2007;37(12):1797-1806. PubMed CrossRef

162. Steffen AM, Gant JR. A telehealth behavioral coaching intervention for neurocognitive disorder family carers. Int J Geriatr Psychiatry. 2016;31(2):195-203. PubMed CrossRef

163. Stein MD, Herman DS, Bishop D, et al. A telephone-based intervention for depression in HIV patients: negative results from a randomized clinical trial. AIDS Behav. 2007;11(1):15-23. PubMed CrossRef

164. Stiles-Shields C, Kwasny MJ, Cai X, et al. Therapeutic alliance in face-to-face and telephone-administered cognitive behavioral therapy. J Consult Clin Psychol. 2014;82(2):349-354. PubMed CrossRef

165. Ström M, Uckelstam CJ, Andersson G, et al. Internet-delivered therapist-guided physical activity for mild to moderate depression: a randomized controlled trial. PeerJ. 2013;1:e178. PubMed CrossRef

166. Taleban R, Zamani A, Moafi M, et al. Applications of text messaging, and bibliotherapy for treatment of patients affected by depressive symptoms. Int J Prev Med. 2016;7(1):46. PubMed CrossRef

167. Titov N, Andrews G, Davies M, et al. Internet treatment for depression: a randomized controlled trial comparing clinician vs technician assistance. PLoS One. 2010;5(6):e10939. PubMed CrossRef

168. Titov N, Dear BF, Staples LG, et al. Disorder-specific versus transdiagnostic and clinician-guided versus self-guided treatment for major depressive disorder and comorbid anxiety disorders: a randomized controlled trial. J Anxiety Disord. 2015;35:88-102. PubMed CrossRef

169. Titov N, Dear BF, Ali S, et al. Clinical and cost-effectiveness of therapist-guided internet-delivered cognitive behavior therapy for older adults with symptoms of depression: a randomized controlled trial. Behav Ther. 2015;46(2):193-205. PubMed CrossRef

170. Uebelacker LA, Marootian BA, Tigue P, et al. Telephone depression care management for Latino Medicaid health plan members: a pilot randomized controlled trial. J Nerv Ment Dis. 2011;199(9):678-683. PubMed CrossRef

171. Ünlü Ince B, Cuijpers P, van ‘ t Hof E, et al. Internet-based, culturally sensitive, problem-solving therapy for Turkish migrants with depression: randomized controlled trial. J Med Internet Res. 2013;15(10):e227. PubMed CrossRef

172. van Bastelaar KM, Pouwer F, Cuijpers P, et al. Web-based depression treatment for type 1 and type 2 diabetic patients: a randomized, controlled trial. Diabetes Care. 2011;34(2):320-325. PubMed CrossRef

173. Vernmark K, Lenndin J, Bjärehed J, et al. Internet administered guided self-help versus individualized e-mail therapy: a randomized trial of two versions of CBT for major depression. Behav Res Ther. 2010;48(5):368-376. PubMed CrossRef

174. Wagner B, Horn AB, Maercker A. Internet-based versus face-to-face cognitive-behavioral intervention for depression: a randomized controlled non-inferiority trial. J Affect Disord. 2014;152-154:113-121. PubMed CrossRef

175. Warmerdam L, van Straten A, Twisk J, et al. Internet-based treatment for adults with depressive symptoms: randomized controlled trial. J Med Internet Res. 2008;10(4):e44. PubMed CrossRef

176. Watkins ER, Taylor RS, Byng R, et al. Guided self-help concreteness training as an intervention for major depression in primary care: a Phase II randomized controlled trial. Psychol Med. 2012;42(7):1359-1371. PubMed CrossRef

177. Watts S, Mackenzie A, Thomas C, et al. CBT for depression: a pilot RCT comparing mobile phone vs computer. BMC Psychiatry. 2013;13(1):49. PubMed CrossRef

178. Williams AD, Blackwell SE, Mackenzie A, et al. Combining imagination and reason in the treatment of depression: a randomized controlled trial of internet-based cognitive-bias modification and internet-CBT for depression. J Consult Clin Psychol. 2013;81(5):793-799. PubMed CrossRef

179. Williams AD, O’ Moore K, Blackwell SE, et al. Positive imagery cognitive bias modification (CBM) and internet-based cognitive behavioral therapy (iCBT): a randomized controlled trial. J Affect Disord. 2015;178:131-141. PubMed CrossRef

180. Wozney L, Olthuis J, Lingley-Pottie P, et al. Strongest Families Managing Our Mood (MOM): a randomized controlled trial of a distance intervention for women with postpartum depression. Arch Women Ment Health. 2017;20(4):525-537. PubMed CrossRef

181. Wright JH, Wright AS, Albano AM, et al. Computer-assisted cognitive therapy for depression: maintaining efficacy while reducing therapist time. Am J Psychiatry. 2005;162(6):1158-1164. PubMed CrossRef

182. Yang W, Ding Z, Dai T, et al. Attention Bias Modification training in individuals with depressive symptoms: a randomized controlled trial. J Behav Ther Exp Psychiatry. 2015;49(Pt A):101-111. PubMed CrossRef

183. Carlbring P, Apelstrand M, Sehlin H, et al. Internet-delivered attention bias modification training in individuals with social anxiety disorder: a double blind randomized controlled trial. BMC Psychiatry. 2012;12(1):66. PubMed CrossRef

184. Pallavicini F, Algeri D, Repetto C, et al. Biofeedback, virtual reality and mobile phones in the treatment of generalized anxiety disorder (GAD): a phase-2 controlled clinical trial. J Cyber Ther Rehabil. 2009;2(4):315-327.

185. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124. PubMed CrossRef

186. Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics. 2000;56(2):455-463. PubMed CrossRef

187. Cuijpers P, Berking M, Andersson G, et al. A meta-analysis of cognitive-behavioural therapy for adult depression, alone and in comparison with other treatments. Can J Psychiatry. 2013;58(7):376-385. PubMed CrossRef

188. Cuijpers P, Donker T, Johansson R, et al. Self-guided psychological treatment for depressive symptoms: a meta-analysis. PLoS One. 2011;6(6):e21274. PubMed CrossRef

189. Motter JN, Pimontel MA, Rindskopf D, et al. Computerized cognitive training and functional recovery in major depressive disorder: a meta-analysis. J Affect Disord. 2016;189:184-191. PubMed CrossRef

190. Chipps J, Brysiewicz P, Mars M. Effectiveness and feasibility of telepsychiatry in resource constrained environments? a systematic review of the evidence. Afr J Psychiatry (Johannesbg). 2012;15(4):235-243. PubMed CrossRef