By Mackenzie Bean 

Hospital quality ratings systems can misclassify hospital performance and provide conflicting information for healthcare consumers, eight physician leaders and researchers wrote in a commentary.

The article's authors assessed the strengths and weaknesses of four major ratings systems on six criteria: potential for misclassification of hospital performance; importance/impact; scientific acceptability; iterative improvement; transparency; and usability.

They also met in person with representatives from each rating system to learn more about their methodologies and challenges. Authors then assigned an average overall letter grade for each ranking system:

-U.S. News & World Report'sBest Hospitals — B

-CMS' Overall Star Ratings — C

-The Leapfrog Group'sSafety Grade and Top Hospitals — C-

-Healthgrades'Top Hospitals — D+

The authors identified similar issues across the four rating systems, including limited data and measures, lack of robust data audits and composite measure development.

Leapfrog CEO Leah Binder responded to the article in an Aug. 14 statement, challenging the authors' assertion that Leapfrog only audits a handful of hospitals submitting surveys to the ratings group.

"Rudimentary fact checking would have uncovered serious errors in the description of Leapfrog's ratings programs in the piece," Ms. Binder wrote.

Mallorie Hatch, PhD, director of data science at Healthgrades, said the article provided a "highly inaccurate portrayal" of its hospital rankings. Dr. Hatch said authors misrepresented their methodology for its overall hospital award and "did not include an analysis of our other service line ratings and awards, which would have addressed many of the criticisms in the article," according to a written statement.

A CMS spokesperson said the agency is confident its star ratings "drive systematic improvements in care and safety." CMS released several methodology changes last spring based on feedback from healthcare stakeholders and said it "looks forward to sharing improvements to the Star Ratings in the future."

U.S. News & World Report, which made several methodology changes to its Best Hospital specialty rankings this year, offered a more positive response to the article.

"We're gratified that the study recognized how responsive we have been to advances in measurement science and feedback from patients, doctors and other stakeholders," Ben Harder, managing editor and chief of health analysis at U.S. News & World Report, said a written statement.