Background: OB/GYN residencies use myTIPreport to improve resident feedback. Validation is viewed as a process with construct validity established from different levels of evidence. In Kane’s validity framework, extrapolation focuses on links between assessment outcomes and other measures of performance.
Methods: Single-institution, prospective cohort study of OB/GYN residents(n=24). Using the ACGME case log system and myTIPreport application, associations of PGY level, number of previous hysteroscopies (HSC), vaginal hysterectomies (TVH), and abdominal hysterectomies (TAH) with myTIPreport ratings of each procedure when performed in the operating room were assessed using Spearman’s test correlation. A pilot study of inexpensive, low technology simulation models was initiated to assess extrapolation evidence for myTIPreport in the simulation setting(n=8).
Results: Associations (p < 0.01) existed between PGY level and myTIPreport scores in the OR (r=0.612 for HSC, r=0.763 for TAH, r=0.864 for TVH). Associations (p < 0.01) existed between number of procedures performed and myTIPreport scores in the OR (r=0.587 for HSC, r=0.681 for TAH, r=0.869 for TVH). An association (r=0.852, p=0.031) existed between number of TVHs performed in the OR and myTIPreport scores during simulation, but not for TAH (p=0.199) nor HSC (p=0.154) due to the small sample size.
Discussions: The association between myTIPreport TVH performance with PGY level and number of TVHs was the strongest, supporting extrapolation evidence. The associations were weaker for TAH and HSC in both the OR and simulation settings secondary to higher ratings for lower PGY levels and fewer cases.
Keywords: Assessment, Evaluation of Clinical Performance, Feedback Evaluation, Milestones, Simulation, Technology
Topics: CREOG & APGO Annual Meeting, 2017, Resident, Faculty, Clerkship Director, Clerkship Coordinator, Residency Director, Residency Coordinator, Patient Care, Systems-Based Practice & Improvement, Interpersonal & Communication Skills, Practice-Based Learning & Improvement, GME, CME, Assessment, Simulation, Faculty Development,
Christopher DeStephano, MD, MPH, Mayo Clinic; Tri Dinh, MD; Brent Seibel, MD; Deanna McCullough, MD; Kelly Best, MD
