How We Make Sure the Rankings Are Right

The new America's Best Colleges 2009 edition rankings go live on August 22. How does U.S. News ensure the integrity of the data and the rankings that we publish?

We have a five-step process:

First step: U.S. News uses definitions of the data that have been developed by experts in higher education to achieve data comparability among schools. The ranking data questions contained in the statistical questionnaires we sent to colleges in the spring and summer of 2008, as in the past, either follow the standardized format in the Common Data Set or conform to definitions used by the U.S. Department of Education or other higher-education organizations.

Second step: After each school submits its statistical data to U.S. News, we analyze the information for factual errors, inconsistencies with other information on its survey, and significant changes that have occurred from what the school reported to us the prior year. How? We send each school a "Data Assessment" report that flags potential errors or problems. If schools have errors or big changes in their data, they must sign off on the changes or correct the errors before U.S. News will use their information in the rankings or publish it.

Third step: After a school clears up all possible problems, we send a final "Data Verification" and ask for each college to do a final check on all the information and for an official at the school to sign a verification form that says the data are accurate and ready for use.

Fourth step: We also cross-check data that the schools have submitted to U.S. News with other official sources. Faculty salaries are cross-checked with the American Association of University Professors; six-year graduation rates with the National Collegiate Athletic Association and the Department of Education's National Center for Education Statistics; and admissions, tuition, financial aid, and financial resources data with the Department of Education's National Center for Education Statistics. In cases where there's a mismatch between the data a school submits to us and another source, we will use the data from the official source and footnote the difference.

Fifth step: As the final but highly important last step, when we calculate the rankings, we do many preliminary runs of the data calculations. This lets us carefully analyze which schools' overall rankings have changed significantly (either up or down) from the previous year, figure out why that happened, and make sure it wasn't because there was a mistake in the data.