Today I will be speaking as one of the invited participants at the U.S. Department of Education’s “Symposium on Postsecondary Institution Ratings Systems,” in Washington, D.C. We all are expected to comment on the methodology and types of data that could be used in President Barack Obama’s plan calling for the creation of college ratings for consumers, if Congress approves a financial aid funding mechanism.
U.S. News welcomes the possibility of Obama’s college ratings, since they could lead to better data for consumers – especially more robust data on student indebtedness, earnings and job attainment after graduation and more comprehensive graduation rate measures – being made available for the first time.
The proposed federal college ratings are not intended to measure academic quality. This is very different from the U.S. News Best Colleges rankings, which seek to determine which schools are tops in undergraduate academic quality.
I plan to stress that now that the department intends to publish high-stakes college ratings and a financial aid funding mechanism, it must use its power to take a much tougher stance and stop ongoing data misreporting by some colleges.
I intend to point out that the department should be realistic – publishing its college ratings could lead to a rise in data misreporting.
That's because the stakes for higher education institutions as they report data to the department will be far higher than in the past, since their data will be tied to institutional funding.
The department needs to enforce the highest data quality integrity standards possible so its college ratings use accurate data and are not subject to data manipulation by higher education institutions.
I also plan to discuss some of the many steps it will take to successfully execute the department’s college ratings project, as it has been described publicly so far. U.S. News publishes many big data projects including our Best Colleges and other educational rankings each year. As a result, we have a lot of experience in this area.
I plan to ask department officials in attendance about how rigid their deadlines are and what that means for the timing of their work. The college ratings are expected to be available before the 2015 school year.
I also intend to bring up many important questions that have yet to be publicly decided, including if outside contractors are going to be engaged to do the work and who will supervise them, which data will be used in the ratings, how the methodology will be created, how the scoring and weighting is going to work, how the consumer interface will be developed and who will test the site and decide if it all makes sense.
The department has said schools are going to be divided and rated by similar mission. I plan to raise the question of whether the department really has the data to enable it to divide schools by similar mission. Why wouldn't the industry-standard Carnegie classifications work? Those have so much credibility and use in higher education.
It’s widely believed that the department will only be able to use the data that the U.S. government is currently collecting to do its initial college ratings. That means that the department will have to use its current information on student debt, earnings, graduation and transfer rates, scholarships, Pell Grants and advanced degrees of college graduates, all of which have significant analytical shortcomings in terms of what the department says it’s trying to measure in its ratings.
I plan to ask how the department plans to handle the problems it will most likely find in its own data when it starts to compare colleges with each other. Finally, I plan to address the difficulties the department will have in
using its existing data to analyze community colleges and institutions that focus on distance education – which mainly enroll working adults, not those
directly out of high school.