Log in  |  Search

Closing Maxwell Part II:
Accountability Ignored and Flawed in a Rush to Close a School

Maxwell HSSince Joel Klein’s new schools do not seem likely to take Maxwell’s neediest students, and since all that its current and future self-contained kids are guaranteed is an opportunity to apply, it seems fair to ask whether or not the community might be well-served by keeping the school open and building on its strengths. This is also a question worth asking because Maxwell is only one of fourteen high schools slated for closing, and the situation I am about to describe with Maxwell is typical of what we find when we look at other closing schools.

The DoE has created two accountability measures to determine school quality: Quality Reviews and Progress Reports.

Quality Review

Quality Reviews are on-site 2-day reviews of the school, performed by a different reviewer every year. For the past several years, the DoE has invested huge resources in its Quality Reviews. Though I don’t know the budget amounts, I do know something about what goes into shaping and implementing them. Over multiple meetings, standards of review are debated and fine-tuned every year. Senior instructional people are trained over several days in order to ensure inter-rater reliability. Reviewers, well-steeped in the philosophy and specifics then perform the reviews armed with standards, rubrics, templates, and on-going support.

Basically — ask anyone in the schools — Quality Review is a very big deal.

In the case of Maxwell, and many of the closing schools, three different reviewers in the last three years found the school to be “proficient.” The Maxwell findings are very detailed and can be found here on the DoE website, but for example, the introduction to the 2008-2009 Review tells us:

Maxwell High School is a place where students feel safe and are excited about coming to school… Additional examples of high expectations include the recently developed systems and structures that teachers use to examine student data and assure usage of this information in class lessons. Regular analysis of formative assessment results has rendered some success, as students were able to accumulate credits towards graduation at a higher rate. Student interest in future professions is realized in career and technical education classes. These effective programs are supported through the school’s small learning community design. These family-like environments enable teachers to collaborate and work cooperatively using common periods to share effective practices.

What is more, according to the Review, Maxwell’s new principal has set the school on a promising course:

The new principal has communicated clearly his high expectations for his students, staff and parents. This message resonates throughout the building both visually and verbally…

…The principal effectively leads the school team in keeping continual student improvement at the forefront of all future planning. While parents are not yet fully engaged in all aspects of school life, staff members are exploring how the use of more current technology, such as text messaging, can keep parents informed. Additionally, school personnel realize the critical importance of ensuring that student needs are met through the provision of instruction that offers a variety of learning opportunities.

None of this means that Maxwell is a perfect school. No school is, and schools like Maxwell that work with very large percentages of needy children need a lot of support and expertise to be successful. But overall, the DoE’s Quality Review found that Maxwell has the people and systems in place to be successful.

The DoE has ignored those inconvenient findings, however, even though schools have been told for years that the Quality Reviews are just as important as the Progress Reports in holding schools accountable. In spite of that, the DoE set aside the review and instead focused on the school’s Progress Report, whose results are highly questionable. I have covered this topic in relation to all the closing schools in general (see here and here), but let’s see how it played out for just this one.

Progress Report Grades: Maxwell and its Peers

In principle the Reports are supposed to produce “outcomes that are not correlated with socioeconomic status, Special Education populations, or other demographic characteristics,” and so, DoE uses a formula to create peer groups of supposedly similar schools. In creating high school peer groups, demographic factors like ELL status and Special Education are considered, along with incoming scores. Schools are then primarily graded by their relative performance compared with these peer schools.

But what was not considered were the challenges of a buried demographic: the percentage of special education students in self-contained classes. Thus, schools with high percentages of these students were held accountable for making the same progress as schools that did not accept them. When they could not keep up, they received low grades and were closed. At Maxwell:

  • All of the forty schools in Maxwell’s peer group serve high-need populations and 38 served special education students. All served significant amounts of students in poverty.
  • However of the forty schools, twelve that received grades had 0-2% self-contained special education populations. I will call these schools the 0-to-2 schools.
  • Meanwhile, fourteen that received grades were at the opposite end of the spectrum and served a population of 10% or more of these students. Maxwell had a self-contained population of about 13%. As anyone in the schools knows, that’s a high percentage. I will call these the 10-plus schools.
  • Four schools were not graded, and the remaining schools with grades had an average of 7.5% self-contained students.

Maxwell and its peers by percent self-contained

  • To be clear: like the other two groups, the 0-to-2 schools did have special education students, but they were not students in self-contained classes. These schools served special education students who need “related services” such as speech, or whose academic, social, and emotional needs indicate that they can thrive in general education without significant extra support.
  • Graduation rates account for 25% of each school’s Progress Report grade. In Maxwell’s peer group (and in the peers for the other closing schools) graduation rates increase as the percent of students in self-contained classes declines. The average four-year graduation rate in the 0-to-2 schools was 63%. The average graduation rate in the ten-plus schools was eleven points lower (51.4%). When we look at the “bottom” half of the 10-plus schools (the 7 schools with the very highest rates (the group that includes Maxwell), five of the seven schools had a grad rate below 48% and the average for the seven was 48.2

Maxwell and its peers graduation rate

As I have said elsewhere (here and here) it is very hard to see this as merely coincidence. The correlation holds for the peer groups of closing schools as a whole. What is more, it is undisputed across the country that self-contained students graduate at much lower rates than other special education and general education students, regardless of the quality of the school or the teachers. One NYC report puts this number as low as 5% based on DoE data, though that also includes students in our District 75 schools, which only serve self-contained students. Whatever the correct number, it is alarming, but what it indicates is that in many cases we need to use data in ways that will tell us what programs have worked with this group of students, rather than shutting down these programs because of political agendas or flawed statistics.

In any case, the graduation rates of the 10-plus and 12-plus schools in Maxwell’s peer group were measured directly against the 0-to-2 schools in order to determine 25% the Progress Report grade. The two groups were also measured against each other for all other aspects of the Progress Reports, with similar results. For example, Regents completion rates, another significant part of the school grade, rise in the same pattern as graduation rates as the percent of self-contained students decrease.

Progress Report Results

The final results on the cards are predictable:

  • Seventy-one percent (71%) of the schools in the 12-Plus group received D’s, thirty-five percent (35%) in the 10-Plus group, and only eight percent (8%) in the 0-to-2 group:

Who got D's?

  • At the other end, seventy-five percent (75%) of 0-to-2 schools were awarded an A, but only 21% in the 10-Plus group.

Who got A's?

  • Alarming as this is, even this result is misleading. DoE probably should have eliminated one of the three A schools in the 10-Plus group altogether. It is a school for deaf children whose challenges may be too different from the other schools to be considered a fair comparison. In addition, that school seems to includes children from sixth grade up, making it impossible for me to determine the rate of self-contained students in the high school. Eliminating that school from the 10-Plus group could make the comparisons even more troubling.

I have taken the time to tell the Maxwell story because it is typical — and not any more dramatic than the story of other schools that the DoE intends to close. In virtually all cases, the schools served high numbers of these students. In virtually all cases, the Quality Reviews were ignored and then the schools were judged by a flawed statistical formula. And in virtually all cases, the DoE has no plan for the students that most need help.