By Cheryl Clark

Starting July 1, medical students who take the U.S. Medical Licensing Examination (USMLE) Step 2 Clinical Knowledge (CK) test will just have to be a little bit smarter.

That's because the USMLE Management Committee has decided that, while the perfect score remains 300, the minimum passing score will rise from 214 to 218. Those scoring below 218 will have to take the test again for a better chance at a residency slot.

In its statement, the USMLE said the committee's pass/fail cutoff decision, which is re-evaluated every 3 to 4 years, was based on four considerations:

- Recommendations from physicians and educators not affiliated with the USMLE who participated in content-based standard-setting panels in March and April

- Surveys of residency program directors, medical school faculty, and state licensing agencies

- Trends in performance of examinees

- Score precision and its effect on pass/fail outcomes

Bryan Carmody, MD, a pediatric nephrologist at Eastern Virginia Medical School in Norfolk who writes frequently about medical education, said the change reflects the fact that medical students are getting better at taking these tests, and getting steadily higher scores. He calls it "score creep."

"You have students who have an incentive to score as highly as possible, and have resources that are optimized to let them score as highly as possible," he said. "It's like you've got to have a car that's better than a neighbor's car."

He noted the evolution and plethora of high-quality online study aids and question banks (Qbanks) have helped the study process a lot, but it's "inadequate in isolation to explain" the increase in test scores.

In a YouTube video Carmody published in November 2023, part of his popular Sheriff of Sodium series, he explained the trend of gradually rising scores for U.S. and Canadian first-time Step 2 examinees has persisted since 1994 when the average score was 200, and the minimum passing score was 167 -- 47 points lower than today. As of the 2023-2024 academic year, the average score was 249.

In his video, Carmody likened it to a runner who breaks the one-mile record, or the development of an electric washing machine instead of the older "scrubbing all their clothes by hand" washboards.

"Scores are rising because students are -- honest to God -- answering more questions correctly now than they did in the past," he said.

To keep up, the USMLE has increased its passing minimum score nine times in the 30 years since, he explained.

He debunked a theory that the questions on the test are easier. In fact, he said in his video, they are getting tougher and more structurally complex. Questions from actual exams decades ago were more commonly short, say 12 words. But over time, he said, the questions have evolved into lengthier vignettes of 50 words, with more information to digest and multiple levels of reasoning required.

Another theory Carmody has heard -- and refutes -- is that students are getting higher average scores because more of them are cheating. First, security procedures have "evolved dramatically," he said. If one thinks cheating is the reason for higher scores, one has to believe that "each successive class of medical students just stayed one step ahead of" the test centers monitoring requirements.

Carmody acknowledged that some overseas testing centers may have had more cheaters, with a center that "goes rogue and starts taking bribes to relax their security procedures or allows ringers to take the exam." But those wouldn't show up in statistics about U.S. and Canadian test scores.

"It's a possible explanation, but it just doesn't fit the data," he said in his video.

Last year, the USMLE reported a pattern of suspicious test results among some test takers and, after investigating, invalidated he scores of 832 Nepali students either for their Step 1, Step 2, Step 3, or all three tests, a finding that put their careers and their visa status in limbo. Of the 832 students, 618 had just one Step 1 score invalidated, 202 had two invalidated, but just 12 had their scores from all three tests invalidated.

Some of the students filed a lawsuit demanding their scores be validated, but a judge denied their request. According to a USMLE spokesperson, most of the 832 chose to accept the USMLE's offer to retake the exams.

Another explanation Carmody has heard for the rising minimum pass scores is that students pay more attention to Step 2 because students get numerical scores, whereas Step 1 results are now given as pass/fail. "But pass/fail scoring for Step 1 didn't begin until the year 2022, and it should be self-evident that anything that happens in the year 2022 is really a pretty poor explanation for a phenomenon" that began nearly 30 years earlier, Carmody said in his video.

Still another theory is that scores are rising because the USMLE has been raising the minimum passing score, and students are reacting to that. But looking at the trend data, he said in the video, it's the opposite. "It clearly seems to be that the examinees are doing better on the test, and then the USMLE is racing to keep up [by raising the minimum passing score]."

He estimated that if 218 were the minimum passing score today, more first-time test takers from U.S. medical schools would have failed, but not that many -- perhaps an additional 1% to 2%.