A Change for the "Challenge Index" High School Rankings

Reader Comments
Back to blog

your results are useful. i need more information abt high school efficiency and performance of these schools. data based.Thanx and best wishes for bright future.

neeru dhir of WA 1:55AM March 07, 2009

How do i find how colleges have ranked particular high schools?

Kathy Butz of OH 2:59PM January 23, 2009

Bob Morse, like many other education journalists, laments the uneven quality of dual enrollment programs in the high schools. However,there are nationally academically accredited dual enrollment programs called concurrent enrollment programs or CEPs, which are college courses taught in the high schools by high school instructors certified as adjunct university faculty. The National Alliance of Concurrent Enrollment Partnerships (NACEP, see www.nacep.org)is the accrediting body and a national forum for concurrent enrollment and other early college credit programs. As the Director of the oldest concurrent enrollment program in the nation, UConn Early College Experience, and the national research chair for NACEP, I would suggest to Bob Morse and others that they investigate the benefits to secondary students who take real college courses in a familiar environment, building confidence and credits, while the secondary instructors and their university colleagues contribute to a reciprocal alignment of curriculum and expectations. Bob could scoop Jay (whom I've been working on for four years) if he can find the right angle to promote the academic quality of high schools partnering with NACEP-accredited programs.

Jill Thorne of CT 3:52PM January 07, 2009

If you count the number of AP tests that were passed (3 and above or 4 and above, depending on whether you are looking for average or elite colleges), and divide by the number of seniors (not the number of test takers), then you have a very simple measure that includes both quantity and quality.
You can't get a good number on that scale by having only a few students taking AP tests and you can't get a good number by just flooding the test with unprepared students.
This was pointed out to Mathews many years ago, but he doesn't care at all about quality---he wants everyone taking AP tests even if they have had no education to prepare them for the tests.
The simplicity of AP passes/number of seniors has all the advantages Mathews claims for measure, while measuring something that parents actually care about.

Kevin of CA 4:18PM December 19, 2008

My post from Eduwonk:
My reading of the U.S. News rankings is that they also give a lot of credit for AP/IB participation without regard to quality. Compare #19 IDEA College Prep with #100 (last) Belmont High:
U.S. News college readiness index: BH = 61.2, ICP = 88.9
U.S. News quality adjusted exams/taker: BH = 3.8, ICP = 1.0
U.S. News exam passing rate: BH = 88.2%, ICP = 17.1%
Enrollment: BH = 1,150, ICP = 231
From reading the IDEA website its clear that the IDEA Prep model is one of students taking IB/AP tests. So it seems IDEA Prep’s ranking is based on its model and not on its results.
I’m guessing that parents who are choosing IDEA Prep are not at all like the parents who are choosing Belmont High. But hey, the ranking is a fun academic exercise!

pm of CA 4:26PM December 15, 2008

I agree with the perspective that Timothy Kanold has shared, but think it is worth clarifying something about the College Readiness Index used to select the top-performing high schools on US News' America’s Best High Schools list.
Full disclosure - I am the lead analyst on the America's Best High Schools project, and developed the method and selection criteria.
Allow me to clarify for Bob Morse, the issue here is not about quality vs. quantity, in that the US News list actually requires quality AND quantity. The College Readiness Index used to rank schools by US News does not create a trade-off between equity/access and excellence/success. This was actually a pretty big concern of ours when we developed the index two years ago.
The index is essentially the combination of two different measures of participation - one that looks at the percentage of students that gain exposure to AP/IB by taking an exam, regardless of how well they do on the exam (access), and the other that looks at the subset of these students not only take at least one exam, but also pass the exam (success). This latter figure is not a passing rate, and thus cannot be manipulated by limiting access. Rather, schools that wish to perform well on this list must do two things well: 1) expose as many students to challenging college-level coursework as possible, and 2) help them to master the content sufficiently that they can pass the exam.
To summarize, it might be useful to compare the two approaches cited here. The Challenge Index actually looks strictly at test-taking, and so measures participation of a sort. The recent change that Jay Mathews has put in place is to essentially create two lists – one ranking the highest rates for schools with passing rates that he has deemed acceptable, and another for schools with really low passing rates (the “catching up” list).
By contrast, the College Readiness Index (CRI) captures both concepts together. Schools cannot achieve high CRIs simply by limiting access to AP and IB courses to their highest performing students. Rather, the incentive under this index is more to offer access to as many students as possible, and ultimately to refine and improve the instruction/delivery and support such that eventually all students not only participate, but master the content. Indeed, many of the best high schools on the US News list are not there because they have lifted all restrictions on participation, and instead emphasize exposing students to challenging coursework as a means of helping them perform to their potential.

Paul Gazzerro of NY 11:26AM December 15, 2008

Both rating lists do have value as Jay Mathews points out. Yet, the last sentence of Bob Morse's comments is the one that highlights the most serious flaw in his criticism of the Challenge Index. In the sentence Bob Morse states
:
"Until Mathews realizes that quality (passing tests) matters much more than quantity (how many are taken), his system will remain flawed and will encourage behavior by high schools that doesn't promote actual student learning"
As an equity issue Morse's statement could not be farther from the truth. Quality does not trump Quantity nor is it an either OR situation. The Goal for high schools in this country is not to " Select and sort" the gifted and talented few - it is to open the doors of access to all students AND get great results as well. The victory will be when we systemically allow more students access to the best curriculum( especially those from under served populations) AND help them to achieve at unprecedented levels. This then would defy conventional wisdom that you can't have both, or that one must come before the other. It is a pursuit of Equity worthy of out time energy and effort.
As an example, imagine a high school of 2400 students where traditionally only 15% of the students are allowed access to the AP or IB curriculum. For a given class then, year in and year out only 90 students out of a class of 600 will even have access to the opportunity to learn, all in the name of "Quality". Thus the inequity is created by denial to the opportunity to learn. If that high school on average has a 90% passing (or quality rate) then about 82 students of the 600 will have passed this rigorous standard. and quality prevails for the very few.
Imagine however, that same High School pursues the the "Cultural Goal" of Access AND Excellence. And allows 50% of it students the opportunity to access parts of the AP or IB curriculum. This now provides access for 300 of 600 students in a given class. Even if the passing rate lowers to 70%, the victory on graduation day is that 210 students (not 82) from this hypothetical class will have had an opportunity to demonstrate the quality we seek. And everyone wins. Jay Mathews is right in that it starts with an intentionality to prepare more students for greater access than ever before. and we can't work on the quality aspect if the students are denied access in the first place.
It has been my experience that students will generally rise to the level of our expectations given the right support, discipline and opportunity. Is this not a pursuit worthy of our best effort?

Timothy D. Kanold of IL 1:52PM December 13, 2008

I have great admiration and respect for Bob Morse's long work at US News and was very happy to see his new high school list appear. It was an intriguing attempt to get at what makes good high schools good, and caused the page views of my list at Newsweek to soar. But we go at this from very different angles, and in both of our cases, our strengths are also our weaknesses. Bob's list makes an interesting attempt to separate schools that raise the test scores of low-income students and those thst do not, but the method is so complicated, and so influenced by the tiniest differences in small samples of low-income students in affluent suburban schools that it leaves many readers in the burbs scratching their heads. Some Montgomery County schools make it and some very similar ones do not, and US News does not release data, at least not that I have seen, showing why. There is also the problem of there being no apparent cnnection between what teachers are doing at those schools and what US News numbers find. What can those educators do to get their school on US News's list. The magazine gives few clues.
The Challenge Index is, as he says, much simpler, and that is our great strength. Those teachers can see exactly what they have to do to make our list--open up their most challenging courses to all students who want to work hard, which most schools still do not do. It separates clearly the schools that have educators who see the worth of that approach and the majority of schools where such activity is considered a threat to the purity of their AP classes. Any parent can get the numbers that dictate the ratings on our list, and do their own arithmetic.
This also allows them to make very interestng comparisons between schools that have significant numbers of students passing the AP tests, and those that do not. It is the transparency of simplicity, and I thinks it helps parents and teachers in ways that the US News list cannot yet do but perhaps may in the future, just as the Challenge Index explores why some schools have significant numbers of students pasing the tests, and some do not.

Jay Mathews of DC 1:32AM December 13, 2008

The number who pass does not matter? Why modify that? It would be okay for under-performing schools to be on the top of the rankings. So what. Somebody has been reading this past methodology for 10 years and thinking they were being informed. No need to rock the boat. Those same people are still out there.

of 5:14PM December 12, 2008