Evaluating vocational rehabilitation programs for substance users: issues in designing and implementing randomized studies

The last decade has witnessed a marked shift in the value placed on vocational services in substance dependency treatment. Because of the welfare-to-work reform movement, stricter work requirements for public assistance, and the elimination of federal disability benefits for addiction alone, patient employment is increasingly seen as an important treatment outcome. Indeed, vocational services for this population are becoming an essential component of the rehabilitation process. Since few evaluations have shown highly positive outcomes, these vocational programs lack an evidence base to guide the choice of service models and components. This paper addresses issues concerning research design and implementation that need to be examined if randomized clinical trials of vocational services for substance users are to yield better employment outcomes. It examines how these issues emanate from the way substance abuse treatment is organized and delivered. The design issues focus on characteristics of service programs that may limit effect sizes and statistical power in evaluation studies. Similarly, the implementation issues concern problems in clinical settings in which researchers may have different goals from clinicians, or may need to supervise certain clinicians in a research study. The paper suggests solutions to these design and implementation problems, including increased sample sizes, controlling interactions between counselor characteristics and counseling methods, developing innovative programs, and engaging service providers in the research process. Although the paper concentrates on vocational rehabilitation, the issues raised are pertinent to other human services as well.