Change to pass/fail score reporting for Step 1



The USMLE program will change score reporting for Step 1 from a three-digit numeric score to reporting only a pass/fail outcome. A numeric score will continue to be reported for Step 2 Clinical Knowledge (CK) and Step 3. Step 2 Clinical Skills (CS) will continue to be reported as Pass/Fail. This policy will take effect no earlier than January 1, 2022 with further details to follow later this year.

In taking this action, the USMLE’s co-sponsors—the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners® (NBME®)—considered the broad range of input received from the Invitational Conference on USMLE Scoring (InCUS), the subsequent InCUS report and its preliminary recommendations, and the extensive feedback and national conversation that preceded and followed the conference.*

In reaching this decision, various score reporting options were considered, including maintaining the status quo (i.e., no change to score reporting). The views of all stakeholders were considered throughout the score reporting discussions. Specific consideration was placed on supporting the educational engagement and overall experience of medical students – and on increasing the dialogue about how multiple assessments of competency could best be utilized by stakeholders in medical regulation and medical education.

The FSMB and NBME believe that changing Step 1 score reporting to pass/fail can help reduce some of the current overemphasis on USMLE performance, while also retaining the ability of medical licensing authorities to use the exam for its primary purpose of medical licensure eligibility. The USMLE co-sponsors also believe that moving to pass/fail reporting of Step 1 while retaining a scored Step 2 CK represents a positive step toward system-wide change, while limiting large-scale disruption to the overall educational and licensing environment.

The FSMB and NBME view this change as an important first step toward facilitating broader, system-wide changes to improve the transition from undergraduate to graduate medical education. The two organizations remain actively involved in the additional work sparked by InCUS, that of a broader system-wide review of the UME-GME transition that is being undertaken by the organizations in the Coalition for Physician Accountability **. As this future work develops, and as additional system-wide changes may unfold, including advances in reliable and holistic assessment of the training of physicians, the USMLE program will offer its resources and partnership.

"The ECFMG supports the informed decision making of the NBME and FSMB Boards on these policy changes. We are looking forward to continually advocating for international graduates as well as participating in the important future conversations on residency selection and transition." – Dr. William W. Pinsky, President and CEO of ECFMG/FAIMER

“The American Medical Association supports the decision to move to a pass/fail reporting structure for the USMLE Step 1 as a first element in improving the UME to GME transition. Our student, resident and physician members voted to endorse a pass/fail policy, in part, because we know our current residency selection system is causing significant distress for our students. The AMA is currently supporting new ideas in the transition from medical school to residency through our funding of the Reimagining Residency Initiative and our Accelerating Change in Medical Education Consortium. We look forward to working with our colleagues across the continuum of medical education in developing improvements in physician education transitions.” – Susan Skochelak, MD MPH, Chief Academic Officer for the AMA

“The transition from medical school to residency training is a matter of great concern throughout academic medicine. The decision by the NBME and FSMB to change USMLE Step 1 score reporting to pass/fail was very carefully considered to balance student learning and student well being. The medical education community must now work together to identify and implement additional changes to improve the overall UME-GME transition system for all stakeholders and the AAMC is committed to helping lead this work.” – Alison Whelan MD, Chief Medical Education Officer AAMC

For more information regarding this policy change, please review the FAQs.

* InCUS (held in March 2019) was sponsored by the American Medical Association, the Association of American Medical Colleges, the Educational Commission for Foreign Medical Graduates, the Federation of State Medical Boards and the National Board of Medical Examiners. For more information, visit https://www.usmle.org/usmlescoring

** Chaudhry HJ, Kirch DG, Nasca TJ, Katsufrakis PJ, McMahon GT, Shannon SC, Ciccone AL. Navigating Tumultuous Change in the Medical Profession: The Coalition for Physician Accountability. Acad Med. 2019 Aug;94(8):1103-1107

Updates


Episode 8: Three Decisions to Result in Future Changes to USMLE - February 12, 2020

pdf iconTranscript

In this episode of USMLE® Connection, Dave Johnson, the Senior Vice President at Federation of State Medical Boards, and Michael Barone, the Vice President for Licensure at NBME, will announce and discuss three recent decisions concerning changes to USMLE.

Summary of Public Comment - October 18, 2019

Exploring Possible Changes to USMLE Score Reporting - July 27, 2019

InCUS webinar now available - June 25, 2019

USMLE Scoring An Ongoing Conversation.pdf - June 13, 2019

Timeline - May 2, 2019

Post-conference statement - March 13, 2019

List of InCUS attendees - March 11, 2019

Episode 7: Updates on InCUS, USMLE Improvements, and Vision - October 4, 2019

pdf iconTranscript

In this episode of USMLE Connection, produced by NBME Communications, Mike Barone, MD, MPH, Vice President of Licensure for NBME, and Dave Johnson, Senior Vice President for Assessment Services at Federation of State Medical Boards, discuss InCUS updates, USMLE improvements, and vision.

Episode 6: The Conversation Continues: Exploring Possible Changes to USMLE Score Reporting - June 13, 2019

pdf iconTranscript

In this episode of USMLE Connection, produced by NBME Communications, Mike Barone, MD, MPH, Vice President of Licensure for NBME, and Dave Johnson, Senior Vice President for Assessment Services at Federation of State Medical Boards, explore InCUS: possible changes to USMLE score reporting.

Episode 5: InCUS Experience and Upcoming Timeline - April 25, 2019

pdf iconTranscript

In this episode of USMLE Connection, produced by NBME Communications, Mike Barone, MD, MPH, Vice President, Licensure for NBME, and Dave Johnson, Senior Vice President for Assessment Services at Federation of State Medical Boards, discuss InCUS Experience and Upcoming Timeline.

Episode 4: InCUS Learnings and Next Steps - April 12, 2019

pdf iconTranscript

In this episode of USMLE Connection, produced by NBME Communications, Mike Barone, MD, MPH, Vice President, Licensure for NBME, and Dave Johnson, Senior Vice President for Assessment Services at Federation of State Medical Boards, discuss InCUS learnings and next steps.

Episode 3: Pros and Cons of Numeric Scoring Vs. Pass Fail - April 12, 2019

pdf iconTranscript

In this episode of USMLE Connection, produced by NBME Communications, Mike Barone, MD, MPH, Vice President, Licensure for NBME, and Dave Johnson, Senior Vice President for Assessment Services at Federation of State Medical Boards, discuss the pros and cons of numeric scoring of the United States Medical Licensing Exam® (USMLE®), pass/fail scoring, and other options.

Episode 2: A High-Quality, Collaborative Approach to Developing the USMLE Program - April 12, 2019

pdf iconTranscript

In this episode of USMLE Connection, produced by NBME Communications, Mike Barone, MD, MPH, Vice President, Licensure for NBME, and Dave Johnson, Senior Vice President for Assessment Services at Federation of State Medical Boards, discuss the collaborative approach to creating the United States Medical Licensing Exam® (USMLE®).

Episode 1: How the USMLE Can Best Serve State Licensing Boards, Students, and Other Stakeholders - April 12, 2019

pdf iconTranscript

In this episode of USMLE Connection, produced by NBME Communications, Mike Barone, MD, MPH, Vice President, Licensure for NBME, and Dave Johnson, Senior Vice President for Assessment Services at Federation of State Medical Boards, discuss the United States Medical Licensing Exam® (USMLE®) program and their thoughts on the Invitational Conference on USMLE Scoring (InCUS).

Continue to: Info »

A National Conversation and Exploration


The Issue
Since its introduction in 1992, the United States Medical Licensing Examination (USMLE) has produced a numeric score in support of its primary purpose as an assessment for initial medical licensure. Throughout the history of USMLE there have been, and continue to be, secondary uses of USMLE numeric scores within undergraduate and graduate medical education.

Many stakeholders value the current practice of reporting a numeric score on USMLE examinations. Over time, however, increased emphasis on USMLE numeric scores as a major factor in residency screening and selection has led others to view this as a negative consequence. Conversely, many in the graduate medical education community value the USMLE numeric score as a common, objective measure of the knowledge and skills of all prospective residents (both US and international graduates).

The Conference
March 11-12, 2019
Philadelphia, PA


The USMLE parent organizations (The Federation of State Medical Boards and The National Board of Medical Examiners), along with the American Medical Association, the Association of American Medical Colleges, and the Educational Commission for Foreign Medical Graduates are convening an invitational conference of key stakeholders to discuss the complex issue of USMLE scores and their use by primary and secondary users within undergraduate and graduate medical education, as well as to explore possible changes to or recommendations (short and/or long-term) for USMLE scoring. While it is anticipated that the conference will focus on numeric score reporting vs. Pass/Fail score reporting, other score reporting options will be considered.

Representatives from a variety of organizations will participate in the conference, including stakeholders representing a broad spectrum of medical education (undergraduate and graduate), state medical boards, examinees, the house of medicine, and the public.

Continue to: The Issue »

The Issue


The United States Medical Licensing Examination (USMLE) is co-sponsored by the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners (NBME). Licensing authorities across the United States rely upon the USMLE to help evaluate the qualifications of individuals seeking an initial license to practice medicine. Three "Step" exams make up the USMLE: Step 1, Step 2 (consisting of the Step 2 Clinical Knowledge [Step 2 CK] exam and the Step 2 Clinical Skills [Step 2 CS] exam), and Step 3. Candidates who have passed all components of the USMLE may apply for a license to practice medicine in the United States and its territories. USMLE delivers approximately 140,000 test administrations annually. International medical students/graduates account for nearly 40% of these administrations.

Students enrolled in an LCME-accredited program typically take Step 1 after completing some of their medical school curricula, Step 2 prior to matching with a residency program, and Step 3 at some point during residency. Examinees receive numeric scores for Steps 1, 2 CK, and 3, which are computer-based examinations. Scores on these exams are considered passing if they are equal to or above the minimum passing scores set by the USMLE Management Committee. Step 2 CS uses standardized patients to test students and graduates on their ability to gather information from patients, perform physical examinations, and communicate their findings to patients and colleagues. Scores from this component are reported as Pass/Fail.

Primary Purpose
An examinee’s USMLE scores are reported to medical licensing authorities for their use in the decision to grant a provisional license to practice in a post-graduate training program and the decision to grant an initial license for the independent practice of medicine. This is the primary intended purpose of USMLE scores.

Secondary Purposes
Trends in medical education have increased examinee efforts to maximize their USMLE performance as demonstrated through the numeric score. These trends include limited residency training slots, particularly in certain specialties, as well as medical schools' adoption of pass/fail grading for foundational curricula. Because residency programs use USMLE scores as a means to screen and select applicants, examinees strive to obtain the highest possible scores.

This use of the score is considered a secondary use of USMLE scores. USMLE was not designed for these purposes and the program did not envision the markedly increased reliance on numeric scores in graduate medical education. Program directors acknowledge the limitations of using USMLE scores for residency selection, but justify the practice given USMLE’s position as a nationally standardized measure of knowledge and skills and as the only common metric by which to evaluate all applicants.

Medical Schools
This emphasis on maximizing USMLE scores has led medical school students to the perception of a "parallel curriculum" to the curriculum of their medical schools. Students engage in their school’s curriculum while simultaneously preparing for what is perceived to be "other material" important for achieving high scores on the USMLE assessments.

In addition, medical school faculty and staff have noted potentially negative effects of preparation for high-stakes testing on student well-being.

Prior Reviews of Scoring Issues
This is not the first time the USMLE program has considered the advantages and disadvantages of numeric score reporting. This topic was discussed and evaluated at the inception of the program (1992) and during a comprehensive review of USMLE (2007-2008). In both cases, the program elected to maintain numeric score reporting. The FSMB and the NBME are keenly aware of the ongoing controversy regarding numeric score reporting and see it as a symptom of the larger challenges in the transition from undergraduate medical education (UME) to graduate medical education (GME).

Continue to: Pros/Cons »

Pros/Cons to Maintaining 3-digit Score Reporting


PROS CONS
USMLE offers highly reliable, objective assessment of relevant competencies. A national standard. A passing score on USMLE demonstrates minimum competency. Pass/Fail reporting suffices for this.
Mitigates the reliability challenges of some medical school assessments Licensure requires only a Pass/Fail outcome
May stimulate student preparation more than a Pass/Fail outcome Focus on numeric scores negatively impacts student well-being
Incremental numeric USMLE performance correlates with other valued measures
e.g., specialty board certification, state board disciplinary actions, improved practice
Standardized test scores best predict other standardized test scores (vs. clinical performance)
Offers a "level playing field" for all examinees, including those from international and new or lesser known schools Maintaining 3-digit score reporting may limit diversity within various specialty programs
If no USMLE numeric score exists, and the demand for a national assessment remains, what fills the gap? Maintaining 3-digit score reporting has an opportunity cost due to students’ heavy focus on maximizing USMLE scores (e.g., less time for research, volunteerism)

Continue to: Deliverables »

Deliverables


It is the intent of the convening organizations to produce several deliverables as part of the work arising from the invitational conference. The first of these is this website, which is envisioned as a primary mechanism for providing updates and status reports on the conference and its outcomes.

Additionally, it is anticipated that the conference will culminate in a statement on the issue of USMLE numeric score reporting. This may take the form of a consensus statement on the part of attendees, a set of recommendation(s), and/or a white paper (informational or a proposal on the issue).

It is also anticipated that a formal paper intended for broader dissemination through peer-reviewed literature may develop from the conference.

Any recommendation(s) to revise existing USMLE policies on score reporting will be sent to the USMLE Composite Committee and FSMB and NBME governance for a final decision or action. We anticipate further details on deliverables and potential timelines to become available after the March conference.

Continue to: Timelines »

Timeline* - updated February 12, 2020


Contact Information for Transcripts
December 2018 Invitations extended to leaders and subject matter experts from major stakeholder groups. These include but are not limited to state medical boards, graduate medical education program directors, student affairs deans, clerkship directors, medical students and residents, and the public. Representatives from organizations such as ACGME, NRMP, LCME, etc. have also been invited to participate.
February 2019 Attendees and organizations asked to share perspectives in advance of meeting
March 11-12, 2019 Invitational Conference on USMLE Scoring
June 2019 Update presented to AMA Council on Medical Education. Preliminary recommendation(s) will be posted on this webpage; 6 week period for open commentary.
Fall 2019 Review of public commentary. Ongoing discussions among co-convening organizations' leadership, as well as FSMB/NBME parent boards and USMLE governance. Presentations at national meetings, including NRMP Transition to Residency Meeting and AAMC Learn Serve Lead 2019.
February 2020 The USMLE parent organizations (FSMB and NBME®) approved transitioning USMLE Step 1 score reporting from a 3-digit numeric score to a pass/faill outcome. This transition will take place no sooner than January 2022.
Late 2020 Further details anticipated later this year

*This timeline is an approximation. The details (both in scope and complexity) of any recommendation(s) derived from InCUS may impact the projected timeline for approval.

Continue to: Literature »

Literature


Overview

The Invitational Conference on USMLE Scoring (InCUS) will provide an opportunity for various stakeholders to review, discuss, and make recommendations about USMLE numeric scoring practices.

In order to facilitate an informed and productive dialogue, a literature review was conducted, focusing on research related to USMLE Step 1 and Step 2 CK score uses and interpretations. Here we provide summaries of pertinent studies published since 2008. The selected articles originate from diverse institutions, and employ varied datasets and methodological techniques.

In general, these summaries speak to various viewpoints and conclusions on the validity argument for Step 1 and Step 2 CK score uses and interpretations. The summaries provide an overview of the current state of available empirical knowledge about the pros and cons associated with contemporary uses of USMLE Step 1 and Step 2 CK scores that in turn will help foster a robust discussion and an empirically-grounded set of recommendations emerging from the InCUS conference.

The research summaries are organized into three thematic sections:

  • Section 1 focuses on studies that address uses and interpretations of USMLE Step 1 and Step 2 CK scores.
  • Section 2 highlights studies that examine relationships among USMLE Step 1 and Step 2 CK scores and measures of specialty-specific educational achievement.
  • Section 3 includes studies that explore uses and interpretations of USMLE Step 2 CS performance.

A fourth category lists a sampling of relevant Commentaries and Letters to the Editor related to USMLE Step 1 and Step 2 score uses and interpretations. The primary purpose for including this list is to acknowledge the perspectives of individuals impacted by Step 1 and Step 2 score uses and interpretations, and to provide examples of the types of conversations in which they are engaged. As these pieces are not empirically based, research summaries are not provided.

To view the research summaries, click on the link associated with the topic area in which you are interested. This will take you to a .pdf file that includes the summaries, as well as links to each of the study abstracts.

Links to Research Summaries and Perspective Pieces