Background: Competency-based, evaluation tools in many of the ACGME milestone areas for OB/GYN are lacking, so we proposed administering an OSCE to supplement existing evaluation tools. Four ACGME milestones (pelvic mass, surgical skills, stress urinary incontinence, and preconception care) were selected for evaluation based on 1) the paucity of existing data on our resident learners on which to base milestone achievement or progression in these areas, and 2) perceived difficulty in obtaining consistent and robust individual assessment of these milestones during routine clinical care.
Methods:
Content experts developed OSCE scenarios and assessment tools locally. Each assessment tool evaluated multiple clinical skills for a milestone, and these skills were assigned a milestone domain corresponding to the ACGME milestone descriptors. The scenarios and assessment tools were internally and externally validated using the Delphi technique. Learners achieved competency for the OSCE and each individual scenario if they demonstrated knowledge of 70% or greater of the skills for that scenario. Additionally, learners passed milestone domains if they performed greater than - 2 standard deviations of the mean for their peer group. Data was analyzed using descriptive statistics.
Results: 36 residents performed the OSCE in the spring of 2016, comprised of 4 separate clinical scenarios. Two independent faculty evaluated each resident on each scenario. Overall, 72.2% of residents were able to achieve competency of all 4 scenarios, the majority being senior residents, compared with junior level residents (77.8% versus 66.7%, respectively). On average, junior learners failed to pass more milestone domains than senior learners (3.6 versus 2.2).
Discussions: Evaluation of OB/GYN ACGME milestone competencies and competency domains can be performed successfully through an OSCE. Individual learners can be compared to their group peers to highlight competency domains that require attention, and overall competency can be assessed and used to supplement existing evaluation tools in assigning milestone progression.
Keywords: Assessment, Communication Skills, Evaluation of Clinical Performance, Feedback Evaluation, Instructional Materials/Methods, Professionalism
Topics: CREOG & APGO Annual Meeting, 2017, Resident, Faculty, Residency Director, Residency Coordinator, Patient Care, Medical Knowledge, Professionalism, Systems-Based Practice & Improvement, Interpersonal & Communication Skills, Practice-Based Learning & Improvement, GME, Assessment, Standardized Patient,
Andrea Shields, MD, University of Minnesota Medical School; Samantha Hoffman, MD; Phillip Rauk, MD
