Toward National Estimates of Effectiveness of Treatment for Substance Use

Objective: To estimate how results would have varied if a substance abuse clinical trial had been conducted with nationally representative adults with substance use and with representative adults receiving substance use treatment.

Methods: Results were analyzed from a multisite clinical trial comparing the effectiveness of the Therapeutic Education System to treatment as usual for outpatient addiction treatment (n = 507). Patients were recruited between June 2010 and August 2011. Abstinence was the primary outcome. The general population sample and general population-treated samples were derived from Wave 1 of the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) (n = 43,093). Propensity scores provided a standardized measure of the difference between clinical trial participants and the 2 NESARC samples. The clinical trial was reanalyzed by reweighting the sample with propensity scores derived from the 2 samples to obtain generalizable estimates of treatment effects.

Results: Before the clinical trial sample was reweighted, the odds ratio (OR) of response to Therapeutic Education System versus treatment as usual in the trial was 1.62 (95% CI, 1.12-2.35). After the sample was reweighted to be representative of the 2 NESARC groups, ORs were 1.33 (95% CI, 0.34-5.26) for the representative sample with any substance use and 1.64 (95% CI, 0.82-3.27) for the representative treated sample.

Conclusions: Applying propensity score weighting to clinical trial results provides a method for estimating the population generalizability of clinical trial findings that relies on effect moderators observed in the study sample and population. Broader confidence intervals in the reweighted samples do not necessarily indicate lack of efficacy of the Therapeutic Education System but rather greater uncertainty concerning effectiveness in general population samples.