Survey disability questionnaire does not generate valid accurate data compared to clinical assessment on an older population

Purpose: Policy-makers and health care managers need accurate information on disabilities in the population to plan appropriate services to the older population. Disability information from population surveys are often very crude as compared to information used on the clinical field. This study aimed to verify the concurrent validity of a survey questionnaire version of a disability instrument (SMAF) administered by lay interviewers, compared to the original SMAF rating scale completed by clinicians. Methods: The survey questionnaire version was developed by a panel of experts and pretested with 15 patients attending a day hospital program using a think-aloud approach. A social worker administered the SMAF rating scale at home to a sample of Community-dwelling frail people over 81 years old who participated in the PRISMA longitudinal study in Sherbrooke, Québec, Canada. The same subjects were contacted one month later by a lay interviewer, who completed the survey questionnaire. The two versions were compared using intraclass correlation coefficients (ICCs) and weighted kappas (WK). Results: 96 subjects participated in the study. The ICC was 0.82 for the total SMAF score and ranged from 0.46 (mental functions) to 0.82 (IADL) for the subscores. The questionnaire version systematically underestimated disability by 4.7 points on average (out of 87) (p < 0.0001). For the case-mix classification generated from the disability assessment (Iso-SMAF profiles), the underestimation was even more striking. Conclusions: A survey questionnaire is not a valid method for accurately estimating disability in a population.