« Back to Search
Assessment of Resident Milestone Competency and Competency Domain Achievement with an Observed Structured Clinical Exam
Background: Competency-based, evaluation tools in many of the
ACGME milestone areas for OB/GYN are lacking, so we proposed administering an
OSCE to supplement existing evaluation tools. Four ACGME milestones (pelvic
mass, surgical skills, stress urinary incontinence, and preconception care)
were selected for evaluation based on 1) the paucity of existing data on our
resident learners on which to base milestone achievement or progression in
these areas, and 2) perceived difficulty in obtaining consistent and robust
individual assessment of these milestones during routine clinical care.
experts developed OSCE scenarios and assessment tools locally. Each assessment
tool evaluated multiple clinical skills for a milestone, and these skills were
assigned a milestone domain corresponding to the ACGME milestone descriptors.
The scenarios and assessment tools were internally and externally validated
using the Delphi technique. Learners achieved competency for the OSCE and each
individual scenario if they demonstrated knowledge of 70% or greater of the
skills for that scenario. Additionally, learners passed milestone domains if
they performed greater than - 2 standard deviations of the mean for their peer
group. Data was analyzed using descriptive statistics.
Results: 36 residents performed the OSCE in the spring of 2016,
comprised of 4 separate clinical scenarios. Two independent faculty evaluated
each resident on each scenario. Overall, 72.2% of residents were able to
achieve competency of all 4 scenarios, the majority being senior residents,
compared with junior level residents (77.8% versus 66.7%, respectively). On
average, junior learners failed to pass more milestone domains than senior
learners (3.6 versus 2.2).
Discussions: Evaluation of OB/GYN ACGME milestone competencies and
competency domains can be performed successfully through an OSCE. Individual
learners can be compared to their group peers to highlight competency domains
that require attention, and overall competency can be assessed and used to
supplement existing evaluation tools in assigning milestone progression.
Communication Skills, Evaluation of Clinical Performance, Feedback Evaluation,
Instructional Materials/Methods, Professionalism
CREOG & APGO Annual Meeting, 2017, Resident, Faculty, Residency Director, Residency Coordinator, Patient Care, Medical Knowledge, Professionalism, Systems-Based Practice & Improvement, Interpersonal & Communication Skills, Practice-Based Learning & Improvement, GME, Assessment, Standardized Patient,