Since the release of our new America's Best High Schools listings, there has been a great deal of focus on the performance of a few high schools in Montgomery County, Md., an affluent suburb of Washington. The key question has been why three high schools in Montgomery County—Walt Whitman High School, Thomas S. Wootton High School, and Winston Churchill High School—that had been gold schools in the 2009 America's Best High School rankings slipped to become honorable mention among Maryland's public high schools in the 2010 America's Best High Schools listings. Two other Montgomery County schools—Walter Johnson High School and Richard Montgomery High School—went from being unranked in previous years to honorable mention in the 2010 rankings.
What is honorable mention status, and why was it included in the America's Best High Schools rankings for the second year in a row? U.S. News believes that being ranked honorable mention is a significant achievement. An honorable mention school:
- is one of the best in country at preparing students to go to college in terms of having a very large percentage of students who take and pass Advanced Placement or International Baccalaureate exams.
- is one that does an exceptionally good job of preparing a vast percentage of its students for college but was not able to meet state test performance criteria. Had these schools met the state test criteria, their college readiness performance would have placed them among the top 100 schools nationwide, or high enough for gold level status. That means they would have scored high enough on the college readiness index to earn a gold medal but didn't fully meet Step 1 or Step 2 as described in the America's Best High Schools methodology.
U.S. News thought it was very important that schools with such high levels of college readiness receive some kind of recognition, while at the same time maintaining the rigorous standards embedded in the methodology to keep these schools separate from the gold medal winners. Only 37 high schools in 12 states received an honorable mention medal. Receiving honorable mention means that these five Montgomery County high schools are top schools.
Still, the question remains: How did these five Montgomery County schools that were honorable mention but had college readiness high enough to be gold fall short in the 2010 rankings? Here's why the numbers worked out the way they did:
According to Paul Gazzerro, the lead analyst at Standard & Poor's who worked in partnership with U.S. News to produce the new 2010 America's Best High Schools listings:
". . . as it happens, the reason that some of Montgomery County's district's schools dropped from being recognized (or dropped to being honorable mention) is that they did not meet Step 1 of America's Best High Schools methodology, so I will explain in greater detail how this step works. In Step 1, we first create a state test performance index, and then compare schools to one another based on the statistical relationship between these scores and their student poverty rates. The state test performance index takes Maryland High School Assessment test data for math and reading, as reported on the state report card, and weights it according to the percentage of student scoring at each level. 0 points go to the lowest level, 1.0 points to level 2 (proficient), and 1.5 points to the highest level, level 3 (advanced). The resulting index ranges from 0-150, with higher scores being better.
This table shows the index values for Montgomery County, Maryland's high schools for the past two years, along side the recognition status that U.S. News assigned in the previous two America's Best High Schools rankings. Note that the 2009 list used data from 2006-07, and the 2010 list (just released) used data from 2007-08. As you can see, all schools improved their state test performance index substantially from one year to the next, but some more so than others. This was true statewide in Maryland. U.S. News then takes these index values and plots them along with each school's student poverty rate (typically the percentage of students receiving free or reduced-price lunches). While the statistical correlation is not perfect, it is by far the strongest predictor of performance and allows the methodology to create a level playing field for schools by effectively comparing them more to their peers with similar poverty rates than to the state as a whole. The resulting regression line, or expected performance, can be compared to the actual performance index. In order to take into account measurement error to define exceptional performance, we then draw a confidence interval of plus/minus one standard deviation around this expected value. Only those schools that performed at a level that was more than one standard deviation better than expected were considered to have met this Step 1 criterion. To present this, we created a risk-adjusted performance index, and only values that equal or exceed 1.00 (as in the one standard deviation) meet the Step 1 criterion.
For a visual representation of Maryland schools compared on this risk-adjusted measure, please go to page 27 of the Methodology Technical Appendix-State by State Guide (pdf). Additionally, this table shows Montgomery County, Maryland's high schools risk-adjusted performance index values, along with the state test performance index explained above and the percentage of students who are economically disadvantaged (the latter is a measure of student poverty representing the percent of students receiving free or reduced-price lunch). This table is sorted by risk-adjusted performance index, highest to lowest. You'll notice that some high schools came close to be greater than 1.0, the value needed to pass Step 1, but only one high school in Montgomery County—Montgomery Blair High School—met the Step 1 criterion.
To summarize, the reason that Montgomery County's previously recognized high schools did not meet this criterion is that, while they improved from year to year, other schools in the state improved more, and the bar got higher. This is not to say that these are not very good schools—just that they did not meet the full set of criteria required to be recognized as among America's Best High Schools."
This is only the third time that U.S. News has published the America's Best High Schools rankings. The methodology has evolved each year. U.S. News will continue to analyze the responses and suggestions from a variety of sources, including those in Montgomery County. U.S. News looks forward to starting dialogues with officials from high schools and school systems across the country in order to hear their suggestions for improving the America's Best High Schools methodology.