U.S. News Meets With AACRAO to Discuss Future of College Rankings

I represented U.S. News and participated in a serious discussion November 1 on ways how to improve the college rankings. Hosted by the American Association of Collegiate Registrars and Admissions Officers (AACRAO), the half-day session, held in San Diego, was titled "College and University Rankings Discussion: Can We Improve How We Assess the Undergraduate Experience?" and was part of AACRAO's 2011 Strategic Enrollment Management Conference.

The event follows the recent publication of numerous articles on college rankings in AACRAO's College & University Journal. AACRAO's ranking discussion at the conference focused on one of the journal articles, "Rating (Not Ranking) the Undergraduate Experience: Principles from a National Discussion," by Jason Lane, a professor at University of Albany—SUNY and one of the panelists at the conference.

The event also attempted to "provide historical and analytical perspectives along with thoughtful discussion of how to best provide the public with insightful data about institutions' dedication to, and success at, enhanced student learning." In addition, there was a conversation about the many issues involved in "developing a clear set of principles for college ratings that could lead to meaningful reforms of the current systems through the development of standard institutional assessment clusters and implementation strategies."

AACRAO proposed six principles for the creation of a rating system:

1. Rating without ranking

2. Recognize institutional difference

3. Create common post-admission milestones

4. Transparency through agreement on definitions, data instruments, and collection processes

5. Account for the value-added features of an educational experience

6. Governance by a nonprofit entity

In my presentation at the conference, I gave the U.S. News response to those six principles. U.S. News highly commends AACRAO and those involved with developing these principles since these goals would be very important for higher education to implement and for prospective students to have access to this level of comparative information, which does not exist today. U.S. News will not stop doing numerical rankings, since we believe the data can support that type of sequential rankings. U.S. News also noted that for-profit entities, such as U.S. News, are very successful at reaching consumers compared to nonprofits and that nonprofit management doesn't equal success.

However, the rest of AACRAO's principles are up to the higher education community to achieve or develop independently. Why? In most cases they are standards or concepts that are not primarily geared to what a publisher like U.S. News can accomplish unilaterally or should do as an independent media organization. Some of the ideas at this point in time are aspirational—including goals such as creating post-admission milestones and measuring value-added data—because there currently aren't agreed-upon standards or ways for schools to gather, publish, and report outcomes and value-added data so it can be used to compare colleges.

Arguably some of these principles are already being followed to some degree by U.S. News in our Best Colleges rankings, including recognizing institutional differences and having a transparent system that uses common definitions and data instruments.

Some of the other principles and goals are basically asking colleges to operate very differently than many of them do today in order to emphasize, measure, and publicize what really should be measured and what really should matter according to the principles. Finally, U.S. News offered to set up an AACRAO/U.S. News advisory board in order to have ongoing conversation about how to improve the rankings.

Presenters:

Jason Lane, Ph.D., assistant professor of educational administration and policy studies at the Rockefeller Institute of Government, University of Albany—SUNY

Mark David Milliron, deputy director for higher education, Bill and Melinda Gates Foundation

Media panelists:

Robert Morse, director of Data Research, U.S. News

Doug Lederman, editor, Inside Higher Education

Paul Glastris, editor in chief, Washington Monthly