Ensuring the College Rankings Data Are Correct

The 2011 Best Colleges rankings go live on August 17. How does U.S. News ensure the integrity of the data and the rankings that we publish?

We have a five-step process we use before the rankings are published.

Step 1: U.S. News uses standardized and accepted definitions of the college data that have been developed by experts in higher education to achieve data reporting comparability among schools. . As in the previous years, the ranking data questions contained in the statistical questionnaires we sent to colleges in the spring and early summer of 2010 either follow the standardized format in the Common Data Set or conform to definitions used by the U.S. Department of Education or other higher-education organizations.

Step 2: After each school submits its statistical data online via a password-protected U.S. News website, we analyze the data for factual errors and inconsistencies with other information on that school's survey. In addition, we check to see whether any large changes had occurred from what the school had reported to us in the prior year. We do this by sending each school a "data assessment" report that flags potential errors or problems with its data. If schools have errors or big changes in their data, they have to sign off on those big changes or correct the errors before U.S. News uses their information in the rankings or publishes it.

Step 3: After each school clears up all possible problems, we send them a final "data verification" report and asked for each college to do a final check on all its information and for an official at the school to sign a verification form indicating that the data are accurate according to the definitions and are ready for U.S. News to use.

Step 4: We also cross-check data that the schools submitted to U.S. News with other official sources. Faculty salaries are cross-checked with data from the American Association of University Professors; six-year graduation rates are compared with data from the National Collegiate Athletic Association and the Department of Education's National Center for Education Statistics; and admissions, tuition, financial aid, and financial resources data is checked against information from the National Center for Education Statistics. In cases of a mismatch between the data a school submits to us and another official source, we will use the data from the official source and footnote the difference. We also obtained annual alumni-giving data from the Council for Aid to Education and used it in the rankings, if schools failed to report alumni giving.

Step 5: As the final but highly important last step, in mid-summer when we start crunching the numbers to produce the final Best Colleges school rankings, we do many preliminary runs of the data calculations. This lets us carefully analyze which schools' overall rankings had changed significantly (up or down) from the previous year, figure out why that happened, and make sure it wasn't because there was a mistake in the data.

After these steps are complete, we are ready to publish the rankings.