An evaluation of the validity of inferences made from 3 diabetes assessment instruments: A Rasch analysis

BackgroundMany patients who have diabetes lack adequate knowledge, positive attitudes, and sufficient resources to achieve optimal outcomes in the management of their disease. A key to assessing the impact of pharmacists' interactions with diabetes patients is the resultant impact on patient knowledge, attitudes, and satisfaction. As such, it is important that quality instruments be used to obtain accurate and dependable measures of these outcomes.ObjectiveTo evaluate the validity of inferences made from 3 separate diabetes instruments used in the assessment of patient knowledge, attitudes, and satisfaction.MethodsThis pilot study enrolled 30 patients with diabetes mellitus to evaluate the following 3 instruments: (1) Check Your Hemoglobin A1CIQ, (2) the revised Diabetes Questionnaire, and (3) a satisfaction questionnaire. The instruments were used to assess patient knowledge of diabetes and diabetes-related complications, attitudes toward having diabetes mellitus, and satisfaction with the services provided in a pharmacist-run diabetes clinic, respectively. Rasch analysis was used to determine if the instruments were able to measure the concepts they are intended to measure when used in this sample.ResultsAfter evaluating the 3 instruments, it was determined that Instrument 1 displayed construct underrepresentation and some mistargeting. Moreover, Instrument 2 demonstrated reasonably good rating scale function but exhibited construct underrepresentation and ceiling effects. Finally, Instrument 3 did not meet the necessary requirements for proper rating scale function and displayed ceiling effects and mistargeting.ConclusionsThis pilot evaluation suggested that none of the instruments were useful in this population, which reinforces the need for researchers to use item response theories to examine the psychometric properties of instruments used in reporting various patient outcome measures. Pharmacists and other health care professionals should be alerted to potential problems with the validity of inferences made from underperforming instruments, so as to prevent inaccurate conclusions.