In other situations, a large percentage of the schools in the most dire category of NCLB improvement, called "restructuring," were not identified as schools for improvement the next year. A school that did not meet its AYP for six consecutive years was placed into restructuring.
In Arizona, Massachusetts, Nevada, Rhode Island and South Carolina, at least 50 percent of the schools removed from improvement were in corrective action or restructuring under NCLB, the report says.
But according to Hyslop, such a large percentage of those schools were removed once their state waivers kicked in because those states had higher-than-average shares of those types of schools to begin with. It could also be the case that states added more holistic factors to their accountability systems, such as college and career readiness indicators or student growth measures, which make student performance look much better than before.
Moving forward, Hyslop says both individual states and the Department of Education need to work to collect more extensive data looking into what changes states have made to their accountability systems to determine whether the right schools are making the cut.
And such data could be particularly useful as 34 states and the District of Columbia begin their waiver renewal processes in January and February 2014, although that data is not explicitly required.
But Hyslop says it's unlikely that such data will be available in time.
"Just the sheer numbers of how many schools aren't identified are pretty terrifying ... but I think the more important question is asking whether those are the right schools," Hyslop says. "So I hope that states start to focus more on which schools are identified and whether they're the right ones, and we actually have data and evidence to show that they are."
Updated 12/17/13: This article has been updated to include comments from Randi Weingarten, president of the American Federation of Teachers.