Feasibility of patient and peer surveys for Maintenance of Certification among diplomates of the American Board of Anesthesiology

•The ABA pilot tested MOCA Patient Care Survey and Peer Survey among oral examiners.•Response rates were 15% and 75% for patient survey and peer survey, respectively.•Both surveys showed high satisfaction, not discriminating among physician performance.•Only one-third of the physicians identified areas in their practice that need improvement.•We recommend exploring other approaches to improve benefits of physician assessment.

Study objectiveThe initial developmental standards for Maintenance of Certification programs proposed by the American Board of Medical Specialties included the administration of patient and peer surveys by the diplomate every 5 years. The aim of this pilot study was to determine the feasibility of Maintenance of Certification in Anesthesiology Program (MOCA) patient and peer surveys in a selected group of American Board of Anesthesiology (ABA) diplomates.DesignThe design was a pilot test of survey instruments—MOCA Patient Care Survey and MOCA Peer Survey.SettingThe setting was the ABA, Raleigh, NC.SubjectsThe subjects were ABA-certified anesthesiologists who were active examiners for the primary certification oral examination as of January 2013.MeasurementsFifty-one participating physicians in the patient survey group distributed brochures, which included a link to the MOCA Patient Care Survey, to up to 100 consecutive patients at the point of care. Fifty-one participating physicians in the peer survey group distributed invitations to MOCA Peer Survey via e-mail to 20 peers in a variety of roles. Participants developed and evaluated a practice improvement plan based on survey results. Participants were also surveyed on their opinions on the feasibility of implementing the piloted survey instrument in their practices.Main resultsResponse rates for the patient care and the peer surveys were 15% and 75%, respectively. Both surveys indicated a high level of satisfaction with the diplomates; approximately two-thirds of physicians could not identify practice areas in need of improvement.ConclusionsThese results suggest that threats to the validity of these surveys include distribution bias for peer surveys and response bias for patient surveys and that surveys often do not provide actionable information useful for practice improvement. Alternative approaches, such as including anesthesiologists within an integrated institutional evaluation system, could be explored to maximize the benefits of physician assessments provided by peers and patients.