U.S. News & World Report sent the embargoed version of its 2023 “Best Grad Schools” list to law schools a few weeks ahead of its official release and one admissions expert said there appears to be significant changes to the methodology that could result in increased “volatility in the rankings.”
Mike Spivey, of Spivey Consulting, wrote in a blog post on his website Tuesday, “[W]e can confirm that there have been changes to the methodology used to calculate schools’ overall ranking” that are set to be released to the public March 29.
First, U.S. News has increased the weight allocated to its bar passage metric. Previously, the bar passage score was assigned a 2.25% weight; that has been increased to 3%, according to Spivey’s blog.
Second, U.S. News has changed how the bar passage metric is scored. Previously, it considered a school’s first-time bar passage rate only in the jurisdiction where a majority or plurality of its graduates took the bar, comparing that rate to the jurisdiction’s overall average pass rate, according to the blog.
With the new methodological changes, U.S. News now considers first-time bar passage rates in every jurisdiction where a school’s graduates took the exam, which means that schools that send their graduates to many different states will now have a more representative result in the bar passage metric, according to the blog.
“This may also be bad news for schools in states like California—the California bar exam historically results in relatively low bar passage rates,” Spivey wrote, “so graduates who take the bar exam in other states (with higher average overall passage rates) are likely to dilute the school’s performance in this metric.”
The third change U.S. News made was to the library metrics that were introduced in last year’s 2022 edition of the rankings. Previously, there had been seven such metrics, each weighted at 0.25%. Now, U.S. News has consolidated those seven into a single metric, “the ratio of full-time equivalent professional librarian positions to students. That single metric is weighted at 1%,” according to the blog post.
“We expect that these changes, and the historic 2020-2021 admission cycle, will lead to more volatility in the rankings results this year than is usually the case,” Spivey wrote. “Because the score differences between schools are so small, even minor changes to the methodology can lead to seemingly drastic changes.”
“We would predict possibly two changes in the top six alone but will wait for the public release on the 29th to be sure,” Spivey told Law.com on Tuesday.
A spokesperson for U.S. News said Tuesday that the company “cannot vouch for the accuracy of any information that appears before March 29.”
Last year, the U.S. News rankings that came under scrutiny after undergoing three revisions, the third of which came two days before the official rankings were released. The first revision was to correct a data error on a metric, weighted at 0.25% of a school’s overall ranking, that measures the number of hours each school’s law library is open. The rankings of more than 30 schools changed once that problem was fixed.
The second revision was to account for Asian students in a new, stand-alone diversity ranking after they were initially left off. (U.S. News later decided to postpone the release of that diversity ranking after law deans pointed out that the revised version still didn’t count students of more than one race as “underrepresented.”)
The third revision did away with another library metric that asked how many credit hours were taught by law librarians. Responses on that question, which was new this year, ranged from 10 to 300. That suggested that schools were counting the credit hours in different ways. That metric was also weighted at 0.25% of a school’s overall ranking, so U.S. News dropped the overall weight of the library metric from 2% to 1.75%, while the weight given to the bar pass rate increased from 2% to 2.25%. Once that was recalculated, the overall ranking of 35 schools changed.
https://www.law.com/2022/03/15/law-school-admissions-consultant-this-years-us-news-rankings-may-include-some-major-changes/
