Purpose: To
assess the effect of training and the inter- and intra-rater variability of a
targeting curriculum for ultrasound guided invasive procedures.
Background: We previously demonstrated the content, face, and construct validity of
our curriculum. Here we assess its inter- and intra-rater reliability.
Methods: Echogenic targets (wooden dowels, plastic straws, and wooden balls) were
placed in water filled buckets covered with simulated skin (ballistic gelatin).
We assessed 23 1st year medical students performing two targeting tasks
(guiding the needle to multiple targets in a random sequence), held 4 weekly
one hour training sessions, and reassessed their performance. Completion time
in seconds, errors (as recorded by multiple assessors), and error adjusted time
(5 sec penalty/error) were recorded. These measures were compared before and
after training using a repeated measures t-test. Inter- and intra-rater
inter-class correlation coefficients (ICC) were calculated.
Results: Completion time, errors, and error adjusted time on the dowel tasks were
176±70, 7.4±, 213±95, respectively before training, and 95±61, 2.7±3.4, and 109±74, respectively
after training (all p<0.0001). The
completion time, errors, and error adjusted time on the straw task were 319±98,
9.4±4.6, and 366±114, respectively before training, and 179±57, 3.1±3.2, and
195±67, respectively after training(all p<0.0001). The inter-rater ICC for the dowel and straw
tasks were 0.83 and 0.91, respectively.
The intra-rater ICCs were 0.93 and 0.90, respectively.
Discussions: Conclusions:
Student performance on the targeting tasks improved markedly after 4 weekly
training sessions. The inter- and intra-rater reliability was excellent. This
provides additional validity evidence for our novel ultrasound guided invasive
procedure task targeting curriculum.
Topics: CREOG & APGO Annual Meeting, 2018, Resident, Faculty, Residency Director, Patient Care, GME, CME, Assessment, Simulation, Maternal-Fetal Medicine,
Joshua Nitsche, MD, PhD