Focus
Student Scene
November 11, 2002
 Steve Simon Photo by Graham Ramsay |
Clinical Exam Scores May Predict Future Performance on Boards
"Evaluating clinical competency is now a central focus in medical education at HMS and around the country," said Steven Simon, HMS assistant professor of ambulatory care and prevention, who recently led a study to test the correlation between scores on physical diagnosis in Objective Structured Clinical Examinations (OSCEs) and performance on the U.S. Medical Licensing Examination Step One. He and colleagues found a modest correlation between students' OSCE scores and Step One of the USMLE and a strong correlation between scores in two skill areas.Though they differ in their emphasis, both exams require students to interpret and integrate data to solve problems. The findings suggest that OSCEs hold promise as a valid and reliable tool to predict other outcomes such as clerkship scores, advanced USMLE results, and specialty-certifying examinations. The study also supports the early use of OSCEs as educational and evaluation instruments in medical school.
The Need for OSCE
Introduced in 1992 to replace the Federation Licensing Examination, the three-part USMLE series emphasizes competency in applying medical knowledge. Step One specifically measures an examinee's ability to use fundamental biomedical science to correctly interpret data, identify microscopic specimens, and solve clinical problems. It does not, however, test skills pertaining to patient interaction and physical diagnosis, a pedagogic void HMS fills with an OSCE. This exam takes place at the end of the required Patient-Doctor II physical diagnosis course.Designed in 1994 at HMS, the OSCE serves to identify the strengths and weaknesses of skills that students have learned and practiced through history-taking and physical examination with select patients. During one of these clinical exams, students spend two and a half hours rotating through 16 clinical stations. Lasting either six or 12 minutes, each station is composed of nine formats, used alone or in combination: question and answer, preceptor role play, standardized patients, actual patients, mechanical or structural models, 35-mm slides, audiotape, videotape, and CD-ROM. Preceptors at each station observe students carrying out the assigned task for two thirds of the given time and give feedback in the remaining time.
The Correlation
The study gathered data from a total of 355 second-year students who, from 1998 through 2000, attended one of 10 OSCE sessions during one week in April. The findings, which appear in the September Medical Teacher, found a strong positive correlation between total scores from the OSCE and USMLE Step One on skills in two areas, identification of abnormality and differential diagnoses, which are most predictive of the overall Step One exam results.The researchers analyzed student performance by total OSCE score, station scores, and a skills score. The study team included Suzanne Fletcher, HMS professor of ambulatory care and prevention at Harvard Pilgrim Health Care; Kevin Volkan, former director of medical student assessment; Claus Hamann, HMS instructor in medicine at Massachusetts General Hospital; and Carol Duffey, former HMS registrar.
--Trang Au