Reliability of the 2005 Middle School Youth Risk Behavior Survey

PurposeTo evaluate the reliability of the middle school version of the 2005 Youth Risk Behavior Survey (MSYRBS) questionnaire.MethodsA convenience sample of 232 Midwestern seventh and eighth grade middle school students completed the MSYRBS questionnaire twice in a 2-week period (14 days apart). The MSYRBS questionnaire, which queries a variety of health risk behaviors, was administered in a manner that preserved anonymity but allowed Time 1 and Time 2 matching. This was accomplished by using two questionnaire scantrons coded with the same unique number, and destroying all used materials to ensure that each participant was matched with their code. Kappa statistics were calculated for individual questions and group characteristics using SAS.ResultsThe mean kappa was 62.6% and the median was 66.5%. Kappa statistics for each item ranged from −2.4% (injection drug use) to 83.8% (suicide contemplation). Negative kappa values were found for two items that had extremely small cell sizes. Kappas did not differ by gender, grade, or race. Based on nonoverlapping confidence intervals, there were no items that had significantly different prevalence estimates at Time 1 vs. Time 2. Nine items (24.3%) and one category (alcohol-drugs) had kappas below 61.0%.ConclusionsThis preliminary study suggests that the reliability of the MSYRBS is high over time. A number of items should be further examined to determine whether they should be amended or omitted from future versions of the MSYRBS. Further research with larger and more diverse samples is recommended, potentially involving children as content experts.