In a recent entry in the Concurring Opinions blog , written by George Washington law professor Daniel Solove, titled How to Fill Out the U.S. News Law School Rankings Form, he ponders the serious question about how academics should fill out the reputation survey, which accounts for 25 percent of the overall law school ranking methodology.
Solove writes, "Every year, U.S. News compiles its law school rankings by relying heavily on reputation ratings by law professors (mainly deans and associate deans) and practitioners and judges. They are asked to assign a score (from 1 to 5) for the roughly 200 law schools on the form. A 5 is the highest score and a 1 is the lowest. While many factors that go into the U.S. News rankings have been criticized, the reputation ratings, by and large, are considered one of the best components in the rankings system. But should it be? Does anyone have any advice for our poor dean? How are people to fill out the U.S. News ranking forms in good faith to accurately reflect their sense of law school reputations?"
The widely read Wall Street Journal's Law Blog also weighs in on the topic of the U.S. News law school reputation survey in a recent post: Is the U.S. News Ranking Methodology Too Simple?
Blogger Ashby Jones writes, "The U.S. News & World Report annual ranking of U.S. law schools has become the legal academy's favorite punching bag. Every year, about this time, folks start criticizing the survey's methodologies, reiterating how easy it is for schools to game the system. (At the same time, few can question the survey's importance.)"
Both blog posts asked me to weigh in on how the U.S. News law school reputation survey works, why we only use a five-point scale, and whether the system U.S. News uses offers enough choices for raters. U.S. News believes that using a five-point scale is appropriate for the level of knowledge that respondents at each law school have about other schools.
It should be remembered that U.S. News is averaging the ratings of all the respondents for each law school. The results for each school are published at the one decimal point level, not at the whole number. Therefore, the averaging produces a great deal of granularity in the final academic reputation results. Because the overall U.S. News law school rankings are based on a combination of 12 factors, we believe we have enough varied indicators of academic quality to produce rankings that accurately show the comparable standing of one law school versus other law schools.