Do Law Schools Report Their Data Honestly?
Just how honest are law schools when they report their data to U.S. News for our 2010 Best Law Schools rankings? Each year, we ask law schools to report the same data to us that they report on the American Bar Association's annual accreditation questionnaire. It turns out the schools are pretty reliable in their data reporting. (Of course, there were some notable exceptions and data errors that I have written about in my blog: Updates to Some Grad School Data and What Happened With Brooklyn Law School.)
The basis for this assessment comes from a study by Tom Bell, a law professor at the Chapman University School of Law in Orange, Calif. He has just published A Model of the 2010 USN&WR Law School Rankings on his blog, Agoraphilia.
Professor Bell says of his work:
"As in every year since 2005, I this year again built a model of the law school rankings published by U.S. News & World Report. Figuring out the rankings—the '2010' rankings, as USN&WR's calls them—proved especially trying this time around. USN&WR changed several parts of its methodology this year and the ABA, which distributes statistical data on which my model depends, fell far behind its usual publication schedule. Finally, though, the model ended up generating scores gratifyingly close to those that USN&WR assigned law schools."
Here's a snapshot comparison of the results of the real U.S. News law school rankings and Bell's simulation of the U.S. News Top 100 law schools.
To what degree did Bell's model duplicate the U.S. News law school rankings? Bell tested its accuracy using "r-squared," a statistical test that measures how close one set of numbers is to another set. It shows that his 2010 edition rankings achieved an r-squared score of .999—in other words, his results were nearly identical to the U.S. News rankings for the scores of those schools in the Best Law Schools' Top 100.
The fact that Professor Bell was able to duplicate our methodology and outcomes using data collected from a different source proves that the U.S. News rankings process is very transparent and accurate.
Professor Bell concludes:
"For now, I'll just offer this happy observation: The close fit between USN&WR's scores and the model's scores suggests that law schools did not try game the rankings by telling USN&WR one thing and the ABA (the source of much of the data used in my model) another. Even a skeptic of law school rankings can find something to like in that."
Tags: law school | rankings
Tools: 
Share
|
| Comments (11) | Print
Reader Comments
Chicago #1? I don't think so...
Chicago may be slightly underrated by the USNWR rankings. Its reputation rankings by peers, judges and lawyers put it just below Harvard, Yale and Stanford, where it should be. Fantastic school, but to say it "should be #1 or #2" is just wrong. So is saying that "they will never break the top 5." Chicago has been ranked fifth or higher, several times over the years. Finally, their acceptance rate isn't lower because students applying to Harvard and Yale "don't know about it." Anyone who has the numbers and has done the research to apply to Harvard and Yale knows about Chicago, and almost everyone that applies to law school these days applies to at least six schools. Further, Chicago offers fee waivers, which the top-3 schools don't, making the number of applications to those schools even more impressive.
Chicago is a fantastic institution all-around, and I agree with you about it being one of (if not the) most rigorous undergraduate programs in the country, better educationally than a lot of Ivies. But let's not kid ourselves--Yale was the best long before these rankings came out, and would be if USNWR was to go under tomorrow.
There's no way to rank law schools!
Look. Most American law schools are good at what they do. And I doubt seriously that all of these students who apply to the Harvards and Yales would be clamoring to get into those schools if the rankings did not exist. As one who has been admitted to school, I can tell you with complete certainty that I picked my school because it was the highest ranked. All eight of the schools I got into were very good schools, but, in the end, the pressure to take into account the "lay prestige" of the highest ranked school was just too great. Even more disturbing, I knew after final seat deposits, that I had made the wrong decision, so I chose to defer. Luckily, the school I committed to is allowing me to apply to other schools this year.
I can also tell you that some of the so-called second and third tier schools do a much better job of training students for actual practice, while the "elite schools" act as though every one of their students is headed to SCOTUS. Do they really have their heads that far up their butts?
Morse...your rankings are a lie, and if I had my choice, I would go to one of the top second tier schools. But I just don't have that luxury. In the end, however, I am going to pick according to "needs" and "fit" this year, instead of ranking.
I agree
Steve, there is actually quite a bit of truth to what you're saying.
advertisement
