The Wait for the National Research Council Rankings Continues
If you are one those in waiting for the National Research Council (NRC) to release its new rankings on U.S. doctoral programs, you will have to be patient a lot longer. The NRC doctoral ranking project, which began in 2003, just released A Guide to the Methodology of the National Research Council Assessment of Doctorate Programs. The guide offers a sophisticated, 191-page explanation of the methodology that the NRC will use when it does publish the rankings, which will cover doctoral programs in 61 doctoral fields at 222 institutions.
The NRC makes it very clear that it is not setting a date for the release of the rankings. Why, after all this time, is the NRC unable to set a firm date for when the new rankings will be published? The NRC's website says:
"With the publication of the Methodology Guide, we have completed a major milestone. The NRC and its committee feel an enormous obligation to produce data and rankings that are of the highest quality possible. Striving for this objective requires careful review of the data and rankings for over 5000 programs. This has already taken considerable time and is nearing completion. The committee must also finish summarizing the most important findings from the data, as well as discuss the strengths and shortcomings of the methodology for incorporation in the final report. The last step of the process is for the final report to undergo the Academy's rigorous review. We are working to complete all this work as expeditiously as possible."
When the rankings finally are published, they will be produced in ranges and not as a single number. Each Ph.D. program will be assessed overall, as well as in depth in three areas: research activity, student support and outcomes, and student and faculty diversity.
Some questions to ponder while we all continue to wait for the new NRC rankings:
Are the data that will be used in the rankings losing their analytical validity since they will be from the 2005-2006 academic year?
Why wasn't the NRC able to produce its rankings more quickly, using more up-to-date information?
How many faculty members have switched institutions and departments since the NRC first started collecting data in fall 2006? This is very important because faculty data are a key part of the NRC's analysis.
Is the NRC's methodology too complicated for the typical Ph.D. student and faculty member to understand? Will the fact that the methodology is very complex make the NRC's rankings less accessible to prospective doctoral students and others in academia?
U.S. News produces the annual America's Best Graduate Schools rankings. In the same period that the NRC has been working on its yet-to-be-completed rankings, U.S. News has published seven rankings using the most up-to-date school data. Since the last NRC rankings were published in 1995, U.S. News has published 14 annual America's Best Graduate Schools rankings. As a result, it should not be a surprise that the U.S. News rankings have become an important source of information for anyone looking to go to a U.S. graduate school.
Tags: graduate schools | rankings
Tools: 
Share
|
| Comments (12) | Print
Reader Comments
Top Ten programs vs. US News Undergrad academic reputation score
This is a more vivid example of the underrating of public universities in comparison to private by US News Undergraduate faculty quality of academic reputation scores(in parenthesis) as opposed the the number of top ten progrsms by the NRC 1995 reports.
School programs in UW News Undergrad
Top Ten(NRC) academic reputation score
Berkeley 38 4.8
Stanford 32 4.9
Harvard 28 4.9
Princeton 22 4.9
MIT 20 4.9
Yale 19 4.9
Cornell 19 4.6
Chicago 18 4.7
Penn 15 4.5
UC San Diego 14 3.8
Wisconsin 14 4.1
Columbia 14 4.7
Michigan 14 4.5
UCLA 13 4.2
Univ Washington 11 4.0
Illinois 10 4.1
Johns Hopkins 9 4.7
Duke 8 4.6
Texas 7 4.1
Minnesota 5 3.9
Georgetown 0 4.0
Washington Univ 0 4.1
Notre Dame 0 4.0
Vanderbilt 0 4.0
REVISE ACADEMIC REPUTATION METHODOLOGY
WHILE THE US NEWS GRADUATE SCORES AND THE NRC GRADUATE SCORES ARE HIGHLY CORRELATED (0.92),AS THEY SHOUULD BE, THE US NEWS GRADUATE SCORES AND THE US NEWS UNDERGRADUATE FACULTY QUALITY OR ACADEMIC REPUTATION SCORES ARE NOT CORRELATE (0.51), NOR ARE THEY CORRELATE WITH THE NRC SCORES (0.52), AND THEY SHOULD BE. THIS PRESENTS A PROBLEM FOR US NEWS AS IT HIGHLIGHTS THE MAJOR FLAW IN METHODOLOGY IN THE US NEWS UNDERGRADUATE RANKINGS.
THE SOLUTION IS SIMPLE...INSTEAD OF POLLING DEANS OF ADMISSION OR CHANCELLORS FOR ACADEMIC REPUTATIONAL RATINGS (THEY KNOW VERY LITTLE ABOUT THIS), POLL HEADS OF INDIVIDUAL DEPARTMENTS:BIOLOGY, CHEMISTRY, ENGLISH, PHYSICS, ECONOMICS, MATHEMATICS, PSYCHOLOGY, SOCIOLOGY, COMPUTER SCIENCE, ETC, AGGREGATE AND TRANSFORM THIS DATA AND USE IT AS THE FACULTY QUALITY SCORE. YOU WILL FIND THAT YOU GET A VERY DIFFERENT AND MORE ACCURATE RESULT THAN WHAT YOU HAVE NOW, WHICH IS AN UNDERRATING OF THE ACADEMIC REPUTATION VARIABLE FOR PUBLIC SCHOOLS AND AN OVERRATING OF THIS VARIABLE FOR PRIVATE SCHOOLS IN THE US NEWS UNDERGRADUATE RANKINGS.
PEOPLE CAN ARGUE ABOUT THE WEIGHT GIVEN TO REST OF THE VARIABLES IN THE US NEWS UNDERGRADUATE RANKINGS, ALL OF WHICH ARE QUANTITATIVE, BUT INACCURATE MEASUREMENTS OF A SCHOOLS REPUTATION IS MISLEADING AT THE VERY LEAST. AS AN EXAMPLE, A SCHOOL LIKE UC SAN DIEGO WHICH IS THE MOST UNDERRATED IN REPUTATION SHOULD HAVE A SCORE OF 4.6 ACCORDING TO THE NRC SCORES AND WHICH NOW IS GIVEN A SCORE OF 3.8, IS A GREAT SCHOOL AND READERS SHOULD KNOW THAT WHEN THEY ARE CONSIDERING SPENDING $140000 FOR A 4 YEAR SCHOOL. THE CORRECT SCORE WOULD MOVE UC SAN DIEGO UP ALMOST 10 PLACES IN THE OVERALL RANKINGS. AS YOU KNOW, SMALL MOVEMENTS IN RANKINGS CAN HAVE A HUGE IMPACT ON THE DECISION OF APPLICANTS AND THE NUMBER OF APPLICANTS FOR ANY GIVEN SCHOOL. IN ADDITION, SCHOOLS LIKE GEORGETOWN, JOHNS HOPKINS, DUKE OR WASHINGTON UNIVERSITY, HAVE ACADEMIC REPUTATIONS WHICH ARE HIGHLY OVERRATED ACCORDING TO THE DATA FROM THE US NEWS UNDERGRADUATE FACULTY QUALITY AND WHEN IT IS COMPARED TO THE NRC(NATIONAL RESEARCH COUNCIL) DATA.
IF YOU LOOK THE PREVIOUS FACULTY QUALITY RATINGS GOING BACK TO 1925, YOU WILL FIND CONSISTENT RANKINGS THROUGOUT THE LAST 85 YEARS, EVEN IF YOU USE THE CURRENT US NEWS AGGREGATED GRADUATE SCORES AS A COMPARISON. BUT, IF THE FACULTY QUALITY OR ACADEMIC REPUTATION VARIABLE SCORES FROM THE US NEWS UNDERGRADUATE RANKINGS ARE USED TO COMPARE WITH THE LAST 85 YEAR HISTORY, THERE IS ALMOST NO CORRELATION WITH THE HISTORICAL DATA.
SINCE THIS VARIABLE ACCOUNTS FOR 25% OF THE TOTAL AND IS THE ONLY QUALITATIVE VARIABLE, THE METHODOLOGY MUST BE ACCURATE AND WHICH, AT PRESENT, IT IS NOT.
US NEWS UNDERGRAD FACULTY QUALITY SCORES VS NRC SCORES
RECENTLY i DID A STUDY ON THE US NEWS UNDERGRADUATE SCORES VS. THE NRC SCORES AND FOUND THAT WHILE THE CORRELATION BETWEEN THE US NEWS GRADUATE SCORES CORRELATE VERY HIGHLY (0.91) WITH THE NRC SCORES, THE US NEWS UNDERGRADUATE FACULTY QUALITY SCORES CORRELATE VERY POORLY(0.52). THE BIGGEST DISCREPENCY IS WITH PUBLIC SCHOOLS WHICH ARE EXTREMELY UNDERRATED (84%)IN THE US NEWS UNDERGRADUATE FACULTY QUALITY VARIABLES AND PRIVATES SCHOOLS ARE ALMOST ALL OVERRATED(75%).
WHEN YOU USE THE NRC SCORES AND SCALE THEM ON A 1-5 SCALE, THE DIFFERENCES BECOME VERY OBVIOUS. FOR INSTANCE, CURRENTLY UC SAN DIEGO GETS A 3.8 ON THE US NEWS UNDERGRADUATE FACULTY QUALITY SCORE BUT GETS A 4.6 WHEN YOU USE A TRANSFORMED NRC SCORE..SOME OTHER INTERESTING RESULTS USING TRANSFORMED NRC SCORES(CURRENT US NEWS UNDERGRADUATE FACULTY QUALITY SCORES IN PARENTHESIS):
UC SAN DIEGO 4.6 (3.8) MOST UNDERRATED FACULTY IN US NEWS UNDERGRAD
WSCONSIN 4.5(4.1)
UCLA 4.5(4.2)
WASHINGTON UNIV. 3.6 (4.1)
UNIV WASHINGTON 4.3(3.9)
MINNESOTA 4.2 (3.8)
NORTH CAROLINA STATE 3.9(3.2)
GEORGETOWN 3.5(4.0)
NOTRE DAME 3.5(4.0)
YALE 4.7(4.9)
VIRGINIA 3.9(4.3) ONE OF THE FEW OVERRATED PUBLIC SCHOOLS
DUKE 4.2(4.6)
JOHNS HOPKINS 4.1(4.6)THE MOST OVERRATED UNIVERSITY IN US NEWS UNDERGRAD
UNIV ROCHESTER 4.1 (3.6) UNDERRATED PRIVATE SCHOOLS
CAL TECH 4.4 (4.7)
advertisement
