Assessment of service productivity in applied settings: comparisons with pre- and post-status assessments of client outcome

Now there are two approaches to evaluating the effectiveness of human services: repeated assessments of client status on specified dimensions and the assessment of the productivity of services [Green, R. S. (2003). Assessing the productivity of human service programs. Evaluation and Program Planning, 26(1), 21-27.], which requires just one assessment be made when enough time has passed for services to impact the client. The psychometric performance of each approach was compared utilizing routinely collected data about a youth's developmental assets. Of nearly 40,000 youth served in two city-funded, delinquency prevention projects during fiscal year 2001-2, 403 youth from 10 to 22 years of age were assessed with a 7-item service productivity questionnaire in the spring and with a 33-item questionnaire in the spring and the preceding fall. After risk-adjusting the change in level of assets scores, the pre- versus post-assessment approach to measuring change in assets proved slightly more reliable than did the single assessment of service productivity approach. Strong support was found from one project for the construct validity of assessing change in developmental assets with either approach. Minimal improvement in psychometric performance was achieved for risk adjusting service productivity scores over omitting the risk-adjustment of service productivity scores. Service productivity was a better predictor than the pre- versus post-assessment approach of the overall quality of services, and service productivity assessments reflected more positive changes in developmental assets, due to services. It was concluded that an assessment of service productivity not only will conserve funds consumed by evaluations but also will answer questions about the effectiveness of services more directly.