There's a problem with President Barack Obama's plan to develop a universal college rating system: the majority of the data he wants to use are either significantly limited under current law, or do not exist.
Although the more controversial aspect of Obama's college ratings proposal is his plan to eventually tie federal financial aid to college performance, his administration will likely have trouble gathering the data it needs to create those ratings in the first place.
That's because five years ago, Congress passed a provision in the Higher Education Opportunity Act that bars the federal government from collecting student-level data – measures that Obama wants to include in college ratings, such as earnings after graduation, the graduation rates of transfer students, or debt-to-income ratios.
"These problems can be solved by reversing the ban on student-level data and collecting that kind of information that we can use for accountability purposes," says Clare McCann, a policy analyst at the New America Foundation's Education Policy Program.
Supporters of the provision said it would protect student privacy, but many states have begun collecting these kinds of data or submitting them to the National Student Clearinghouse for their own research and tracking purposes. It just isn't available to the federal government, and can't be used for comparison because each state does it a little differently.
The Department of Education, which issues more than $150 billion in student aid to colleges each year, does have access to some data, such as summary reports from colleges and earnings data from financial aid recipients, according to James Kvaal, the deputy director of the White House Domestic Policy Council.
But most of those data are limited to "first-time, full-time" college students that do not represent the entire student population, such as transfer students, part-time students, or those who are returning to school after taking time off.
"What we're missing is a lot of non-traditional students," McCann says. "Those would be the kinds of students whose outcomes might be most compelling for us to look at, and we have no data on any of those."
The data the department does have access to are reported on its College Scorecard, which Obama launched this spring. But that information is also more broad – comprised mostly of average costs at colleges, average student loan debt, class sizes and campus settings – and is not particularly helpful to students who could use more detailed information about college value, McCann says.
The department is unable to provide more specific measures, such as the graduation rates of students who receive Pell grants. Several other aspects are limited under the unit record ban: the department is unable to report how much students earn after graduating or how easily they are repaying their loans.
Kvaal said in a call with reporters on Thursday that Secretary of Education Arne Duncan will be leading conversations with higher education leaders and university officials about "what data might be desirable in the course of developing these ratings."
But McCann says many institutions have a vested interest in keeping such accountability measures out of the public eye and have put up a fight when the department attempted to expand its data collection in the past.