CRU Updates - March 2011

Posted March 17th, 2011

New Item Formats

Since our last update in June 2010, staff has completed successful pilot work in several areas. Test material committees have developed multiple choice questions that assess an examinee's ability to appropriately interpret information presented both in the form of a research abstract and a pharmaceutical advertisement. Additional study is planned, and the items that ultimately appear in the examination may differ from the current design being tested. If planned pilot tests are successful, a small number of these item types will appear in Step 2 CK in August 2011and in Step 3 in March 2012. These dates are tentative; any final decision about the inclusion of these new item formats (and any significant changes to the exam) will be posted to the USMLE website prior to implementation.

Step 2 CS

In 2010, pilot tests of changes to communication skills assessment and changes to the patient note were successful, with further testing planned for 2011. If testing in 2011 is also successful, changes will be implemented in the Step 2 CS examination in mid-2012. Confirmation of any change will be posted to the USMLE website well in advance of its implementation.

Potential changes to the patient note include asking examinees to document their analysis of a patient's possible diagnoses, indicating the evidence obtained from the history and physical examination that supports different diagnostic possibilities.

Envisioned changes to communication skills assessment are targeted to enhance the nature and difficulty of challenges posed by the standardized patient (SP). Examinees will be assessed based on their ability to tailor their questions and responses to the specific needs of the case presented and on their ability to react to the entire range of the patient's concerns. The system of scoring communication skills will also change; while the focus will remain on an examinee's ability to effectively gather and share information, and to establish a professional rapport with the SP, the scoring system used will reward examinees who demonstrate a broader and deeper range of communication skills and approach the encounter in a focused manner. More information about the enhancements to the assessment of communication skills is available in the Fall/Winter NBME Examiner.

Behind-the-Scenes Changes

Other issues being explored in 2011 range from the smallest details (how to code the content and emphasis of each individual question) to the fundamental structure of the USMLE program (how many exams and what scores). Staff is presently engaged in work that will support required changes to the internal systems used to organize and report exam content. When this work is completed (target date: 2013) it will enable new score reports, e.g., competency-based reports to complement traditional discipline and specialty scores.

Over the course of 2011, models of potential changes to the USMLE exam sequence will be reviewed by staff of the ECFMG, FSMB, and NBME; by USMLE governance; and by key stakeholders. Models being considered would maintain approximately the same amount of total testing time across the USMLE exam sequence, and would not increase fees charged to examinees beyond historical trends. Changes under consideration include the following:

  • The number of examinations;
  • The content assessed in each individual examination and in the USMLE program as a whole;
  • The types of assessment challenges, e.g., new item formats described above; and
  • The way that examinees' performance is scored and reported.

Depending on feedback received during this review, plans presently call for public announcement of changes to the USMLE structure in the first half of 2012. That announcement will address both the nature of changes as well as the timeframe for implementation, including the process for transitioning from the current Step system to a new system of USMLE examinations.