Creation and Implementation of a National Emergency Medicine Fourth-Year Student Examination

BackgroundA National Board of Medical Examiners examination does not exist for Emergency Medicine (EM) students. To fill this void, the Clerkship Directors in Emergency Medicine tasked a committee with development of an examination for 4th-year (M4) EM students, based on a published syllabus, and consisting of questions written according to published question-writing guidelines.Study ObjectivesDescribe examination development and statistics at 9 months.MethodsThe committee reviewed an existing EM student question database at www.saemtests.org for statistical performance, compliance with item-writing guidelines, and topic inclusion within the published EM M4 syllabus. For syllabus topics without existing questions, committee members wrote new items. LXR 6.0 software (Applied Measurement Professionals, Inc., Georgetown, SC) was used for examination administration. Data gathered included numbers of examinations completed, mean scores with SD, and point biserial correlation (rpb).ResultsOf the 553 questions assessed, 157 questions met the stated criteria, and 37 were included in the examination. Thirteen new questions were written by committee members to cover all curriculum topics. The National EM M4 Examination was released online August 1, 2011. Nine months later, the examination had been completed 1642 times by students from 27 clerkships. Mean score was 79.69% (SD 3.89). Individual question difficulties ranged from 26% to 99%. Question rpbs ranged from 0.067 to 0.353, mean 0.213 (SD 0.066).ConclusionsA national group of EM educators developed an examination to assess a published clerkship syllabus. The examination contains questions written according to published item-writing guidelines, and exhibits content validity, appropriate difficulty levels, and adequate question discriminatory ability.