# Law Schools Report Accurate Data to U.S. News

December 29, 2010
• Comment (4)

Just how honest were law schools when they reported their data to U.S. News for our 2011 Best Law Schools rankings? Each year, we ask law schools to report the same statistical information to us that they report on the American Bar Association's (ABA) annual accreditation questionnaire. Despite some notable exceptions and data errors over the years, it turns out the schools are pretty reliable in their data reporting.

The basis for this conclusion comes from a recent study by Tom Bell, a law professor at the Chapman University School of Law in Orange, Calif. He has just published "Z-Scores in Model of 2011 USN&WR Law School Rankings" on his blog Agoraphilia.

Professor Bell says of his work:

"As I have for each of the past several years, I this year again built a model of the most recent U.S.News & World Report law school rankings. This year's model matched the published rankings very nicely; comparing the model's scores with the published ones generated an r-squared of .997 (where 1.0 would indicate perfect correspondence). Why do my fellow rankings geeks care about z-scores? In brief, these z-scores measure how well each school performed relative to its peers, thereby establishing its rank. Because U.S. News uses z-scores to rank law schools, so too must any model of its rankings. I weighted these z-scores simply by multiplying the z-score for each school, in each category of data, by the percentage that category influences a school's overall score in U.S. News rankings."

To see the weights and factors Bell used in his ranking calculations, check out the U.S. News Best Law Schools ranking methodology. Bell's blog also includes links to z-scores from previous years.

To what degree did Bell's version of the U.S. News law school rankings model duplicate the actual rankings? Bell tested his rankings model's accuracy using "r-squared," a statistical test that measures how close one set of numbers is to another set. Bell's 2011 edition of the U.S. News law school rankings achieved a nearly perfect r-squared score. In other words, his law school rankings results were nearly identical to the U.S. News rankings for the scores of those schools in the Best Law Schools' top tier.

The fact that Professor Bell was able to duplicate our methodology by using law school data he obtained directly from the ABA proves that the U.S. News rankings process is very transparent and can be duplicated using publicly available data. This exercise also establishes that U.S. News is calculating and weighting the ranking variables as stated in the posted methodology. Users of our law school rankings can be confident that the results are correct given the weights and rankings variables that U.S. News has chosen.

Finally, does Bell's study also prove that law schools are being accurate in how they report their statistical profile data to the ABA, the general public, and U.S. News? No, this only proves that law schools are being very careful to report the same data to U.S. News that they report to the ABA for accrediting purposes.

Searching for a grad school? Get our complete rankings of Best Graduate Schools.

Tags:
ABA,
rankings,
law,
law school

The only thing transparent here is this latest, amateurish attempt at misdirection. Like all big businesses, US News doesn't give a flying #\$%^ about their impact on people's lives; they exist only to make money, and as much of it as possible regardless of the consequences. Speak in the only language they understand - don't buy their product.

News should be truth. Two lies don't make a truth. A lie to the ABA that is then passed along US News does not make US News any less accountable for publishing information they did not attempt to verify. The effect of the information coming from the ABA is much less impactive than coming from US News, as US News knows. Knowing how important this information is to the hundred thousand applicants of law schools every year, US News was reckless in taking the ABA's information without doing any fact checking.

This posting, weeks before the NY Times expose, testifies that US News knew of the lack of accountability behind the information it was using to create rankings. Knowing both that the statistics are fictitious and that their rankings are depended on by hundreds of thousands of people, US News should be held accountable for the debts law school graduates cannot pay back because they don't have jobs.

The only honorable thing US News has left to do at this point is to take responsibility for the lies it has perpetuated. US News needs to print retractions on current rankings and apologize to the law school community for the recklessness with which it acted.

I pledge my support to those who want to take legal action against US News for their recklessness. Thankfully, those US News harmed when turning a blind eye to information they knew was coming from a biased source and using false information to compile what is the by far the most important factor in hundreds of thousands of college graduates' decisions to take on over \$150,000 in debt are at least armed with the tools necessary to implement justice. I think I am covering class actions next semester. I can't wait.

"The fact that Professor Bell was able to duplicate our methodology by using law school data he obtained directly from the ABA proves that the U.S. News rankings process is very transparent and can be duplicated using publicly available data."

That's not the issue, and it never has been. No one has ever disputed the law schools submit the same information to you as they do to the ABA. The problem is that the information they send to both of you is fatally flawed. THAT'S what transparency is about.

#### Morse Code: Inside the College Rankings

Robert Morse is director of data research for U.S. News & World Report and has worked at the company since 1976. He develops the methodologies and surveys for the Best Colleges and Best Graduate Schools annual rankings, keeping an eye on higher-education trends to make sure the rankings offer prospective students the best analysis available. Morse Code provides deeper insights into the methodologies and is a forum for commentary and analysis of college, grad and other rankings.