Making Sure the College Data Are Correct

The new America's Best Colleges 2010 edition rankings go live on August 20. How does U.S. News ensure the integrity of the data and the rankings that we publish?

We have a five-step process before the rankings are published.

First step: U.S. News uses definitions of the data that have been developed by experts in higher education to achieve data comparability among schools and other publishers. As in the past, the ranking data questions contained in the statistical questionnaires we sent to colleges in the spring and summer of 2009 either follow the standardized format in the Common Data Set or conform to definitions used by the U.S. Department of Education or other higher-education organizations.

Second step: After each school submits its statistical information online via a password-protected U.S. News website, we analyze the data for factual errors and inconsistencies with other information on each school's survey. In addition, we check to see whether any large changes have occurred from what the school reported to us the prior year. How? We send each school a "data assessment" report that flags potential errors or problems with its data. If schools have errors or big changes in their data, they must tell us that those big changes are OK or correct the errors before U.S. News will use their information in the rankings or publish it.

Third step: After a school clears up all possible problems, we send a final "data verification" report and ask for each college to do a final check on all its information and for an official at the school to sign a verification form that says the data are accurate according to the definitions and ready for use.

Fourth step: We also cross-check data that the schools have submitted to U.S. News with other official sources. Faculty salaries are cross-checked with the American Association of University Professors; six-year graduation rates with the National Collegiate Athletic Association and the Department of Education's National Center for Education Statistics; and admissions, tuition, financial aid, and financial resources data with the National Center for Education Statistics. In cases where there's a mismatch between the data a school submits to us and another official source, we will use the data from the official source and footnote the difference.

Fifth step: As the final but highly important last step, when we start crunching the numbers to produce the actual school rankings, we do many preliminary runs of the data calculations. This lets us carefully analyze which schools' overall rankings have changed significantly (either up or down) from the previous year, figure out why that happened, and make sure it wasn't because there was a mistake in the data.

After these steps have been completed, we are ready to publish the rankings.