APGO

Katharyn Meredith Atkins, MD
Beth Israel Deaconess Medical Center, Boston, MA

Study Objective: Peer Observation of Teaching (POT) is important in the development of educators. For POT to be effective it should be criterion based and utilize reliable instruments. In the September 2011 Clerkship Review of the Harvard Medical School (HMS) OB/GYN Clerkship, the Course and Clerkship Review and Evaluation Committee recommended incorporating a POT program to improve faculty development. While members of the Clerkship Committee were in favor of pursuing such a program, we felt an instrument appropriate for use in the student didactic teaching seminars did not yet exist. The goal of this project was to develop a peer observation of teaching instrument for use in the HMS OB/GYN Clerkship Didactic Seminars.

Methods: This project utilized a modified Delphi process. The Delphi process is an iterative, multistage group facilitation technique, designed to establish group consensus. The HMS OB/GYN Clerkship Directors at all four HMS teaching sites as well as key educators served as the experts in this process. Key educators were previous Clerkship Directors at HMS sites with extensive knowledge of the clerkship.

Results: To compile the initial list of criteria for the teaching instrument, all HMS Peer Observation of Teaching Instruments were reviewed. In addition, previously published instruments were reviewed. 32 criteria were generated from this list. Participants were asked to rate the criteria using a scale of 1 to 4 with 1 being very important and 4 indicating the criteria should be eliminated. Measures of central tendency, mean and standard deviation (SD) were used to determine the level of group consensus. For the initial round, 10 participants rated 32 criteria. A mean of less than or equal to 2.0 and SD less than 0.99 were used to determine inclusion in the instrument. In the second round 9 participants rated 19 criteria and a mean of less than or equal to 1.56 was used to determine criteria for inclusion in the instrument. All SD were less than 0.99 in this round. In the final round 10 participants rated 15 criteria. In this round there were two criteria with means great than 1.4 and SD greater than 0.5, and these items were discarded. Additionally, two criteria were combined, leaving 12 criteria in the final instrument.

Conclusions: A modified Delphi process resulted in a strong consensus on criteria to be used in a peer observation of teaching instrument for use in the didactic teaching seminars in the HMS OB/GYN Clerkship. The next step in this project is to perform a pilot study of the instrument to evaluate internal consistency and inter-rater agreement. Ultimately we hope to establish a formative POT program in the OB/GYN Clerkships at HMS.

Key Words: Peer Observation, Teaching