Purpose: To compare milestones assigned to PGY 1 and 2 Residents via
an Oral Milestone Exam versus the traditional retrospective monthly electronic
evaluation system to assess how they aligned.
Background: Programs are tasked with implementing assessment tools
to evaluate the 28 milestones. Most programs use some form of an electronic
evaluation at rotation completion. The Clinical Competency Committee
reviews all information for final score assignment each six month period.
Methods: In 2015, we instituted an Oral Milestone examination
to assign the six-month milestones and compared those scores to our
retrospective monthly on-line evaluations. We evaluated PGY 1 and 2 residents
in a simulated forum on milestones, which included Medical Knowledge, Patient
Care, and Interpersonal /Communication Skills Competencies. All residents were
given simulated patients, cases, and/or skills while each examiner was given
the specific ACGME milestone assessment sheet to score. The residents were
provided with immediate feedback.
Results: From 2015-2018, 78.4% of PGY 1 and 43% of PGY 2 residents
scored higher on the real-time oral milestone exam. Additionally, in 82% of PGY
1 residents and 52% of PGY 2 residents score on the oral exam was at
0.5-1milestone level higher than the retrospective electronic monthly
evaluations.
Discussions: Clinical Competency Committees are tasked with Milestone
assignment to all residents every six months. Evaluation tools that most
reflect the actual milestone completion is a mission of all programs. We set
out to assess whether our electronic monthly retrospective evaluation system
was mirroring the assessment performed on our residents with the real-time oral
milestone exams at the end of the six month interval, just prior to submission
to the ACGME.
Our data suggests discrepancy in our online retrospective milestone evaluation
versus the real-time assessment of an oral exam. Not only did residents score
higher in most circumstances in an oral format, but they were higher by a
half-whole milestone level in the majority of the cases. It would suggest that
our ability as educators to recollect the performance of a resident at an
interval later than the performance may be flawed.
Programs may want to consider instituting an oral milestone examination for
enhanced milestone assessment.
Topics: Resident, Faculty, Osteopathic Faculty, Residency Director, Patient Care, Medical Knowledge, Professionalism, GME, Assessment, Simulation, Standardized Patient, General Ob-Gyn,
Kourtnie McQuillen,
MD, West
Virginia University, Department of Obstetrics and Gynecology; Rahul Mannan,
MS 4; Robert
Shapiro, MD;
Mahreen Hashmi,
MD