Editor's Comments: Type II Reviewing Errors and the Search for Exciting Papers

Second-year medical students performed a head-to-toe screening physical examination on trained patient instructors (PIs) as a high-stakes examination before beginning clinical clerkships. PIs completed a 138-item checklist and instructed the students in the proper performance of any incorrectly executed or omitted items. To assess the accuracy of the PI ratings, 'monitor' PIs watching the encounter by video completed checklists for 11 retests of failed students in 2001 and for 28 randomly selected encounters in 2002. Checklist-level discrepancies measured by mean absolute checklist difference were 8% in 2001 and 9% in 2002. Case-level mean difference between raters in 2002 was -2.9%. The inter-rater reliability for a single rater, estimated by the intra-class correlation coefficient for total encounter scores, was 0.95. The assessment and feedback were well received by the students. The Head-to-Toe examination provides a reliable assessment of a student's ability to carry out a complete screening physical examination. Patient instructors can be trained to acceptable accuracy and reliability despite the length and complexity of the exercise, and can provide effective feedback and teaching targeted to individual student deficiencies.

• 저널MIS Quarterly · 32(2) · 415–9
• 토픽Evaluation and Performance Assessment · Management Science and Operations Research
• DOI10.1080/01421590410001696452
• 저자Detmar W. Straub