Aim: How can the ability to identify and understand basic propaganda techniques empower you to make better informed decisions?

Do Now: Read and briefly discuss with a partner “Recognizing Propaganda Techniques and Errors of Faulty Logic”

Motivation: How many of you have ever been excited to purchase an item, partake in an activity, or follow a course of action, only to find yourself disappointed by the outcome? Who would like to share the situation? Elicit a response or two. Why did you specifically make the decision that you did? What led you to make the decision?

SWUT: The best way for the individual not to be manipulated into making decisions not in their best interest is to understand propaganda techniques.

Step 1: Briefly discuss and elicit examples of the propaganda techniques found in Cuesta College’s “Recognizing Propaganda Techniques and Errors of Faulty Logic”

Group Task/Differentiation:

1. Have students work in groups of four or five to do a close reading (pen, pencil, and/or highlighter in hand to underline main ideas of each paragraph, write notes, and/or write questions/comments on doc) of DOE “Turnaround” doc, dated January 13, 2012.
2. Have roughly half of the students in each group answer the following questions:
   1. (p.1) What evidence of additional support have you seen this year? Have support services and or programs increased or diminished this year?
   2. (p.2) Based on the context of this paragraph, how does Walcott define meaningful system? Does the paragraph imply that the current evaluation system is meaningless?
   3. (p.3) Does the misuse of the plural possessive form in the first sentence imply Walcott is performing poorly in his educational duties? Why would the specific “conditions the UFT insisted on” be left out of this document? Is there any evidence to show the replacement teacher would better serve our students?
   4. (p.4) Is “real accountability” clearly defined? If so, what does it specifically mean? If not, why not?
   5. (p.5) Does Bloomberg currently have the authority to carry out his plan? Why would the DOE hold back the details of their plan?
   6. (p.6) What specifically will be used to screen the existing staff? What are “rigorous standards for student success”? Why is the term “significant portion” used? Is there an insignificant portion?
   7. (p. 7) Does the DOE currently have the authority to carry out their plan? Is the approval of the plan presented here the only course of action in restoring the funding? What does the phrase “whatever it takes” mean? What details are provided to clarify “best equipped”?
   8. Using one sentence, clearly describe the DOE’s plan to improve your educational experience at John Dewey High School.
3. Have the other half of students in each group use the DOE source doc to try to identify examples of the four propaganda techniques listed in “Recognizing Propaganda Techniques and Errors of Faulty Logic”
4. Have the groups discuss and share out their answers.

Closure: Exit slip: Other than the “turnaround” model, decide what other solutions are possible?

HW: What appropriate and viable means are available to students to influence major decisions made regarding their education?

It starts with this chart (and then gets worse).

Even though DOE claims that the Progress Report grades are demographically neutral, DOE did not fail a single high school with lowest concentrations of high-need students (that top 1/3 in dark green).¹ And, though the D’s and F’s are spread across the bottom 2/3 (in blue and red), it was overwhelmingly the D’s and F’s with the highest needs that made the “pre-engagement” list — the short list from which DOE would ultimately choose the final closures. 65% of the highest-need D’s and F’s were put on the short list, but only 15% of the schools in the middle where the students on average had fewer challenges to overcome.

And it gets worse.

Because to be on the short list only means that you might or might not close. Once they create the short list, the DOE claims it “reviews the school data, consults with the superintendents and other experienced educators who have worked closely with the school, and gathers community feedback.”

That’s what they say, and it is certainly true that they make a good show of it, running from school to school and having all sorts of sympathetic meetings. But in the end? Take a look at which ones land on the final list.

So then: half the D’s and F’s fall into the middle of the needs spectrum, but only 15% of them made the short list — and none of those will close. Not so the schools with the highest need students, where 40% of the D’s and F’s are slated to shut down.

But even within that bottom third there are variations in need. Here are all the schools that made the short list, school by school, arranged by the level of need. The higher the bar, the better prepared students are when they arrive in high school. Red schools on the final list for closing. Blue schools are not.

Keep in mind, these schools are arranged by need level, not performance level. So, for example, Gateway (third from the right), got a higher Progress Report score than every school on the short list except for one.² But Gateway ranks among the neediest in the city. It is also one of the few new schools to take high-need special education students in comparable numbers to the older city schools. So, it’s closing.

All five of the neediest schools are closing, but six of the seven least needy are staying opened.

And here is one more comparison. This time let’s take a look at the three schools on the far ends of the spectrum — the schools that are will not close on the left, and the schools that will on the right. Here are different challenges the schools face.

The closing schools have higher levels of poverty, special education students, high-need special education students, overage students, and boys. Concentrations of special education and overage students are factored into the DOE Progress Reports. Poverty and gender are not.

The difference in the number of boys is especially worth noting. Most schools on the short list have about as many girls as boys, but Gompers is 77% boys, virtually all of whom are black and Hispanic. Graduation rates for black and Hispanic boys citywide is over 10 percentage points lower than the graduation rates for black and Hispanic girls. Gompers also has the highest citywide concentration of self-contained students, other than two schools already closing. Legacy and Gateway are not far behind. I cannot find the city’s graduation rate for self-contained students, though I have heard it is in the single digits. Special education rates in general, are at 30%, city wide.

Why do I bring this up? Because it shows how differences in the levels of school challenge can be presented as differences in school quality. DOE doesn’t factor the rates of boys and girls into its Progress Reports, but even worse, it justifies shutting schools like Gompers by citing those rates out of context. Graduation rates at Gompers are “in the bottom 1% of high schools” says DOE, failing to mention, however, that the school it is also in the bottom 1% of all schools when it comes to need, the only other schools joining it there being a handful of schools already shutting down. And it’s in that bottom 1% even before you factor in the extraordinarily high numbers of boys, and the extraordinary number of boys that are high-need special education boys. Last year, the DOE shut down a school that was 100% boys, citing low graduation rates. Maybe the rates were low compared to a citywide average. But for the population, which was 10% self-contained and 100% boys? For all we know, it was doing about as well as Stuyvesant, given the obstacles confronting so many of the students in that school.

And this is the biggest bone I have to pick with DOE. Should the city do better by its most disadvantaged students? Of course it should. But will it ever do better by them if it continues to harp on the politically convenient claims about “school quality” while failing to speak openly about the overwhelming needs. A few years ago, DOE commissioned a report that basically said that if you concentrate very high need students in very large numbers in schools, the schools would be overwhelmed. At the end the report listed several suggestions going forward, including “constraints on the HS admissions process” that take the needs of students into account — and avoid high concentrations.

But DOE buried that report, and continues to blame the school instead of evolving its school choice model into one that does not disenfranchise students and cause concentrations.

Are there differences in school quality? Maybe there are, but we are not going to find them if we keep measuring — and the punishing — the wrong thing.

¹ School need in all of these charts is based upon the DOE’s calculation of school need, the peer index. For high schools, this includes the average student’s entering score, the percent of students with IEPs and high-need IEP’s, and the percent of student who enter the school overage.

² The Progress Report Letter Grades are based upon number scores. Gateway’s score was 46.

Part of the evidence for this conclusion comes from the conduct of NYC DOE officials during negotiations. Throughout the month of December, the UFT made intensive efforts to bring these negotiations to a successful conclusion before the NYS Education Department’s deadline of December 31. Yet while UFT officers and staff canceled vacation plans to work on a potential agreement, key actors on the DOE side, such as the lawyer who writes up contractual agreements, were outside of New York City on vacation as the clock ticked down.

“I want to welcome all 70 of you to my class. The bad news is half of your teachers have been fired — but I did get a raise!”

To move the negotiations forward, two different UFT-DOE working committees were established, with UFT officers and staff on each committee. The first committee met often, did an extraordinary amount of work, established joint working groups to prepare local assessments and reached agreement in principle on every important issue before it. The second committee, which included two Deputy Chancellors on the DOE side, was an entirely different story. Despite the looming deadline, the Deputy Chancellor leading their side had to be contacted three times before he responded to a request to set up the first meeting of that committee. The DOE group would come strolling into every meeting of the committee at least 30 minutes late. Shortly after the first committee completed its work, the two Deputy Chancellors come to their committee, announced that they would not agree with any system of appeals that was not essentially the same as the status quo, and walked out, declaring the negotiations over despite statements from the UFT that they should continue. President Mulgrew called Chancellor Walcott with an offer to submit this issue to binding arbitration, which was immediately turned down. Within a matter of minutes of the walk out, Tweed release a prepared statement justifying its actions.

Equally telling was the issue over which Tweed broke up the negotiations: whether or not there would be a meaningful system of appeals for end year ratings of ineffective. The DOE has stonewalled UFT requests to provide numbers of appeals filed and sustained under the current U rating appeals system, forcing us to file a Freedom of Information request. The data that we do possess, coming from members who come to us to contest their ratings, suggests a reason why these numbers are treated as ‘state secrets’ at Tweed: of the last 2000 appeals on the UFT’s books, the DOE has sustained the teacher exactly 10 times. The rate at which Tweed’s hearing officers turn down appeals is thus 99.5%, a figure that would be more appropriate for Stalinist show trials than a legitimate due process procedure. That is the process that the DOE refuses to negotiate.

The U rating appeals of the NYC DOE were not always a kangaroo court. Prior to the Bloomberg administration, a meaningful number of appeals led to the overturning of a unsatisfactory rating, a sign that hearing officers actually examined the facts presented to them. But under Bloomberg, the hearing officers have been under marching orders to turn down all appeals. It is this change, combined with the burden of proof that the new state evaluation law places on a teacher receiving two ineffective ratings in a row, that has led the UFT to insist upon changes in the current appeals process.

Appearing on his weekly radio show with John Gambling, Mayor Bloomberg explained why he opposed the UFT’s position that teachers should have the right to appeal negative unsatisfactory and ineffective ratings to an independent hearing officer, rather than a DOE employee.[1] “The principals’ job is to decide who’s good, who’s bad,” the mayor said. “It’s their judgment, that’s their job.” Subjective ratings are simply the way things work with bosses, and a mix of good and bad personnel decisions are “just part of the real world.”

Before we take the mayor at his word, it is worth recalling what he was saying about principal judgment on personnel matters nine months ago, when tenure decisions were being made. Conveniently ignoring the fact that tenure decisions are made at the end of a three year probationary process that involves the dismissal and voluntary resignation of ineffective novice teachers, such that approximately only 1 in every 2 new teachers achieved tenure, Bloomberg argued that the rate at which principals had been recommending tenure at the end of probation over the past few years, in the 90% range, was unacceptably high. In school after school, principals were ordered by superintendents to change positive recommendations for awarding tenure into deferrals. Clutching as a trophy the decline of awards of tenure to 58%, Bloomberg crowed that “we’ve turned what had been a joke interpretation of the state law, to make it something that you have to work hard, earn, and show that you are better than the average bear” to get.

So why is a 90% rate of principals recommending tenure, at the end of probation “a joke,” but a 99.5% rate of turning down U ratings appeal perfectly acceptable? Simply because the first is a positive evaluation of teachers, while the second is a negative evaluation. So long as principals are putting notches in Mayor Mike’s belt for fired teachers, their judgment should be treated as next to infallible, but when they offer positive evaluations, they will be overruled in a second. Nowhere was this clearer than in a U rating appeal decision the UFT recently had overturned in an Article 78 legal proceeding: on the record, the principal had explicitly said, again and again, that she was not contesting the teacher’s appeal, but the DOE hearing officer still upheld the unsatisfactory rating. It took a court to do the obviously right thing.

The bottom line here is that Tweed’s vision of a good teacher evaluation process is not one in which decisions are made on the basis of sound educational judgment, but one which delivers a requisite quota of dismissed teacher scalps. If you doubt it, consider the misleading comments a Deputy Chancellor began to make over the past summer, that 20% of all teachers had been rated ineffective in Tweed’s Talent Management pilot that has been practicing observations using the Danielson Framework for Teaching. For the last half year, at meeting after meeting, the UFT has been asking the DOE for the study that supports these claims, all to no avail.[2] We have concluded that the study has not been shared because it does not exist: the DOE has simply decided that 20% is a good target for the numbers of ineffective ratings, and so the claim continues to be made and to appear in DOE PowerPoint presentations.

At the end of the day, one conclusion is inescapable: Mayor Bloomberg decided that he had no intention of negotiating in good faith with the UFT over the subject of teacher evaluations. The plan was always to blow up the negotiations required by law, with a strategy of then trying to pressure Albany to change the teacher evaluation law and allow the DOE to continue its kangaroo court U rating appeal process. From the beginning of this process, he and his devotees at Tweed were acting in bad faith.

[1] In its public statements, the NYC DOE has claimed that the UFT wanted an independent hearing officer for both ineffective and developing ratings. This claim is, quite simply, a fabrication out of whole cloth. The UFT has asked for the independent hearing officer only for the ineffective rating, as it alone can lead to dismissal and the loss of livelihood.

[2] Where reputable, independent scholars have studied the use of Danielson framework for lesson observations, such as the study of the framework’s introduction in Chicago public schools by the University of Chicago’s Consortium On Chicago School Research, the rate of ineffective has varied from 3% to 6% annually.

New York City is losing its teachers.

More than 66,000 have either resigned or retired since Mayor Bloomberg took control of the schools.

Teachers leave one of the toughest jobs in New York City for a variety of personal and professional reasons, but the most common single reason is a lack of support from supervisors and the Department of Education.

Teaching is a craft that is acquired over time, and teachers desperately want to improve their skills. That is why the United Federation of Teachers led the campaign to create a better teacher evaluation system, one that put a priority on helping all teachers do their job better. The UFT’s role was critical in creating the new system, and in going to Washington, D.C. to help get federal funds for it through the Race to the Top program. Starting last spring, many of our members with expertise in evaluation worked for months on the state subcommittees designing the new system.

We have been trying to work with the Bloomberg administration to iron out the final details of the new system, but the administration has refused to engage in meaningful talks about teacher and principal improvement. Instead it has focused on ensuring that administrators have unlimited power over their employees. If we agree, it will mean that supervisors’ decisions can never be properly reviewed, much less overturned. This would be true even if their negative rating of a teacher or a principal can be proven to be the result of their refusal to inappropriately change a student’s grade or to give students credit for courses they have not properly completed.

Make no mistake about it. The administration has put tremendous pressure on principals to make their schools appear to be successful. But any claims of success ring hollow in the light of national tests that show very limited student progress for the system as a whole, and state measures that show that while the high school graduation rate is increasing, the number of graduates ready for college is only about one in five.

The sad truth is that Mayor Bloomberg’s “reform” agenda — raising class size across the system; closing schools and “warehousing” the neediest students; pushing art and music out of the schools to make room for more test prep; turning a deaf ear to parents’ concerns; and appointing a completely unqualified publishing executive to be Chancellor — hasn’t made our schools better.

A real teacher evaluation system that helps all teachers improve while providing checks and balances is a critical step toward stopping the hemorrhaging of our teaching force and making our schools more effective. At the same time it would help ensure that teachers who cannot succeed in the classroom leave the profession.

We have an open offer to the administration to continue our negotiations on this issue, or even to take it to binding arbitration. It’s time the administration sat down with teachers and principals to come up with an agenda that will actually help our children learn.

Sincerely,

Michael Mulgrew
President
United Federation of Teachers

DOE says: “Un-screened high schools opened since 2002 continue to earn higher grades and have better graduation outcomes than un-screened high schools opened before 2002.”

First of all, the new schools are not unscreened. Under Bloomberg, the DOE instituted what it calls a ‘limited screen” policy, and that policy does not work well for many at risk kids. Limited screening gives first preference to students who have actively made themselves “known to the school.” After that, the preference goes to any student in the borough, rather than to kids from the neighborhood. To be known to the school students must attend an admissions fair or have put themselves forward by some other means. Like lottery admission systems, limited screening tends to bias in favor of families that are engaged in the process. In the case of limited screening though, that bias is exacerbated by the fact that the screening is embedded in a complex process that students must navigate, wherein they choose 12 schools and rank them. For a better understanding read Darwin or, more simply, see two New York Times articles, here and here.

The schools are screened in another way as well. Historically, the DOE’s high school admissions catalogue shows that the new schools did not offer the small, special classes required for self-contained special education students, the group of students that struggle most in school and are most likely to influence school outcomes (see here and here). By excluding the classes from their admission pages, the DOE effectively excluded the students. This year, under pressure, the DOE changed its wording, and we shall wait to see how that goes.

In any case, the new schools are not “unscreened.”

As far as “better outcomes” goes, once the news schools reach high concentrations of high-need students, they have a so-called failure rate that is the same as older schools and that includes the low graduation rates. My post from a few weeks ago is about exactly that. New schools also have lower rates of college readiness when compared to older schools.

DOE says: “Graduation rates at new schools are higher than the schools they replaced.”

Here DOE is comparing new schools to the old school that used to exist in the same building. Graduation rates might be higher, but the populations are not the same. Here, for example, is a chart showing Columbus, which is closing, and its replacement schools. As the population of self-contained students rises, the school’s grade on the Progress Report declines.

Graduation rates parallel the Progress Report grades, rising as the percent of self-contained students decreases. Global Enterprise, the new school with higher populations and a C is also closing.

And here is JFK (another closing school) and its replacements:

I was able to chart Columbus and JFK because these schools are still phasing out, and so the data is available. In its press release, however, DOE chose a much earlier example, where public data is not available. Let’s look at that.

DOE says: “… the new schools we have opened on the Van Arsdale campus in Brooklyn had a graduation rate of nearly 83% in 2010 – 38 percentage points higher than the 2002 graduation rate at Van Arsdale High School. The new schools on the Van Arsdale campus are achieving these results with a similar population of students that were served by the school they replaced.”

These sound like wonderful schools on the Van Arsdale campus, and the anecdotal evidence I can gather suggests the same. But wonderful or otherwise, the replacement schools are not similar to the school that they replaced.

For starters, like all new schools Van Arsdale’s replacements had limited-screened admissions that gave preference to students who had actively engaged in the admission process. And remember those admission catalogues I mentioned, where the policies effectively excluded students who needed self-contained classes? Well, all three of the schools that replaced Van Arsdale have followed that policy and are serving no self-contained special education students. They are also serving the lowest percentage of overage students in the Van Arsdale neighborhood. In fact, all three schools show up in the top half of the DOE’s need (“peer”) index. In other words these three schools have lower needs than at least half of the schools in the city (they are at the 75th, 58th and 57th percentile). Virtually all NYC high schools serve high need students, but depending on which of these three schools you mean, between 250 and 300 schools have higher needs.

So how does that stack up against the now closed Van Arsdale? DOE selected an example where direct records on the student population are not publically available, but there are some things we know. When DOE moved to shut it down in 2002, Van Arsdale was a neighborhood school, serving neighborhood kids in Williamsburg Brooklyn well before gentrification. Soy lattes were hard to come by, and living sustainably meant a roof over your head and regular electricity, not a visit to Whole Foods and organic cotton in the children’s hats. It would absolutely strain credulity to believe that Van Arsdale was serving the same population as the new schools. In fact, the closing schools are virtually always notable for being those with the very highest concentration of high-need students in their neighborhoods, and often citywide. And that would include large numbers of self-contained students.

Old schools and the new schools that replaced them are not the same. For a more thorough discussion of the population differences between old and new schools, see this 2010 report by Jennifer Jennings and Aaron Pallas.

Finally, one more claim by the DOE.

DOE says: “schools earned a higher percentage of A’s and had a higher average percentile rank than non-charters, led by CMO-affiliated schools and charter middle schools.”

This is highly misleading and only true in a technical sense. When DOE made this claim a few months ago, I posted my response about the middle schools and you can read it here. Most of the charter middle schools that got an overall A received that A even though they did not have an A in any category for academic achievement.

In some ways, the DOE press releases have become nearly laughable. Surely they must know, as the rest of NYC knows, that what they are saying isn’t really true, and that they owe all of us a more balanced representation of how our schools are faring under Bloomberg’s watch. But of course, it is not really laughable. DOE is a public institution, not a corporation hawking a product during prime time. It has a responsibility to the public, and I think it would just be really a very good thing for the schools if DOE reported factually, and came clean with the far more nuanced story of what they surely know.

The DOE updated the college-ready information when it released the high school Progress Reports this autumn, so I ran the analysis again. The results are the same, or maybe even worse. College ready rates are low everywhere, but when we break the schools into deciles by level of need, and then compare new and old schools, we see that newer schools are having a harder time getting their students ready for college. Here, for example, are the four deciles that represent schools in the middle of the citywide need range.

And here are all of the deciles except for the 10th where there are too few new schools to make a fair comparison.

In some cases, even the old schools targeted for closing have higher college ready rates. For example, here is a chart comparing college readiness at Lehman High School, with the schools that DOE has identified as similar. The DOE gave Lehman High School an F on its Progress Report, the lowest grade in its group. Yet Lehman has a higher percentage of college ready students than almost every other school. Again, new schools are green and old schools are purple.

Note that nine out of the ten schools with the lowest rate of college readiness are new schools. Meanwhile, Lehman has higher rates than 28 of the 32 other schools with data. The two schools on the far right that outperformed by a rather large leap have no self-contained students in their student body, while Lehman has a population that is 8% self-contained, the highest in its peer group. Because these students face significant challenges that makes even graduation difficult, it is not surprising that schools without these students would have more college-ready students than Lehman.

In all, while the targeted Lehman has 15.2% college-ready, the similar new schools in its peer group average only 6.5%. The old schools (even without Lehman) average 10.1%.

What is true for Lehman is true for other targeted schools as well. Three other old schools targeted for closing (but none of the newer ones) have college ready rates above 10%, and above their similar – school average. And while this is surely not stellar performance, keep in mind that city wide, only about 20% of students are college ready – and that there are 54 schools with under 5% college ready that are not being considered for closing at all.

In any case, citywide , old schools are outperforming new schools when it comes to college-readiness, and it seems hard to justify replacing existing schools with newer ones when the DOE doesn’t know what to do in the new schools once they open them, other than to measure what goes on.

Which is, of course, DOE’s solution to the college-ready crisis: to measure it and use it as a threat. Next year, schools will be “held accountable” in the Progress Reports for higher college-readiness rates, but it is one thing to pressure schools to improve the numbers, and another to focus on educating kids. When DOE wanted high school students to pass more classes, for example, it put credit accumulation into the Progress Reports and voila! — credit accumulation shot right up. But as it turned out, the group of schools with the higher credit accumulation on average (and that was the new schools) was also producing fewer students who were college ready when compared to similar schools. Were students really learning more, or had accountability simply lowered standards and changed the direction of the school system for the worse?

I will skip the possible explanations for the differences between college-ready rates in old and new school results, because I covered that in the summer post, and you can read them there. But as DOE gears up to close and open another batch of schools, Bloomberg and New York City’s leadership need to decide when they’ve finally had enough.

Notes:

The DOE college-ready metric is based on the English and Math Regents tests. Students must earn a Regents diploma, a 75 or higher on the English Regents, an 80 or higher on one math Regents and complete coursework in Algebra II/Trigonometry. Scores of 480 or higher on the SAT can be substituted.

All schools that have both grades and college-ready data are included in this research. The level of need is determined by the DOE’s own need index (the peer index), and there are thirty-three schools in each group. The groups are roughly balanced between old and new schools (with an average of 15 new and 18 old schools per group).

The tenth group, which I did not include in the chart, includes too few new schools to make a fair comparison. There, too, however, the older schools outperformed new schools. Many of the old schools in this group are extremely low-need and some work with the most academically prepared students in the city.

New schools are defined as all schools that opened in 2003 or later.

This year of the 21 high schools have landed on Bloomberg’s latest struggling schools list, at least 8 (38%) are new schools that were opened on Bloomberg’s watch.

And, when you consider that Bloomberg’s new high schools represent about 40% of all existing high schools,¹ you quickly realize that Bloomberg is shutting his new schools at about the same rate that he shuts the older ones. Put another way, this year, DOE is thinking of closing 5.4% of its new schools and 5.8% of its old.

Figure out the sense in that. But if you can’t (because I can’t), read on.

Schools Targeted for Closing

Which new schools are in the crosshairs for closing down? The ones that teach high-need students, of course, and particularly students specifically identified for special focus (students in self-contained classes). As I’ve been saying for a few years now (for example, here, and here), virtually every school that serves these students in higher percentages has become a target for the DOE. Generally, that has meant older schools because new schools do not take self-contained students in large numbers. Old schools close, and new ones open. But now we see that if the new schools educate self-contained students, then the DOE generally labels them as failures too.

Students who are in self-contained classes are unlike any other demographic group commonly used to determine the level of academic challenge a school confronts. When we identify the percent of students who live in poverty, for example, we are identifying a possible learning challenge by proxy. Poor students do tend to struggle in school, but of course not every poor child struggles.

Self-contained students are different. Every single one of these students has been specifically identified by a team of experts to have cognitive or emotional difficulties that significantly impede their learning. Thus, the percent of student who are in small, self-contained classes in a given school gives us a very direct window into a school’s level of challenge. That doesn’t mean they represent the extent of a school’s needs (these schools generally have high rates of poverty as well, for example), but they are a very good place to start.

Of course the DOE does not simply look at the demographics of a school and then label it a failure. Rather, DOE awards each school a letter grade after it runs the achievement data through a formula that is intended to tease out the differences in school performance after the differences in populations have been factored out. Theoretically, all schools have an equal chance of getting good grades. That’s the theory, but in reality, schools with fewer challenges get a lot more A’s and schools with higher challenges get more D’s and F’s. (For statistical details, see this post by UFT analyst Rhonda Rosenberg). Thus, DOE gives the public the illusion that it is taking a sophisticated look at school performance, but, by and large, it’s not.

To illustrate the point consider the 98 high schools where not a single student needs self-contained support. Over three quarters of these schools (78%) got A’s or B’s. Only 5 of the schools got D’s — but none of those D’s are targeted for closing. None of these schools got F’s.

Meanwhile, on the other end of the spectrum sit 19 schools where 10% or more of the students have been identified as needing the special focus of self-contained classes. The Progress Report grades for these schools — and their destinies — are nearly a mirror image of the others. Half of these schools are either already closing, or are newly listed for possible closing. Two more have been targeted for closure in the past (and were saved by the UFT). Four are in danger for next year because they have a C on their Progress Report and have never had above a B.

So:

• 0% self contained = 98 Schools never in danger of closing
• 10% self-contained = 19 schools of which 75% are in danger of closing or already closing

New Schools, Old Schools. No Difference.

I said at the start of this post that at least 8 (or 38%) of the schools that landed on the latest list of “struggling-schools” are new schools. As with the old schools, demographics drive their destiny.

Let’s compare. If we line up all the new schools that are old enough to have received grades, and then we zero in on the 10% that have the highest concentration of self-contained students, we wind up with 15 schools with concentrations ranging between 7.8 and 13.2%. For comparison, we have 32 older schools with self-contained concentrations in the same range. Take a look at the results.

If anyone believes that the policy of closing schools is a winning one, then the chart above should give them pause. Old or new, schools with high concentrations, have high failure rates on the Progress Reports, and slide toward closure.

And it’s not just the school grades that stay the same. Here are the graduation and attendance rates, as well as the survey safety scores from the new and old high-concentration schools. You tell me what’s changed.

It wasn’t supposed to be this way, of course. At the new schools, the principals got to pick their teachers and create their cultures. DOE policies meant that student populations in new schools could not grow beyond a certain size (about 525 kids), and the small size had the potential benefit of making education more personal for students.

So what went wrong?

Conclusion — And A Way Forward

Frankly, what went wrong is pretty much what the UFT said would go wrong. DOE is fascinated by measures, systems, and accountability, and that’s not the same as a fascination with education and real schools. Have you ever had dinner with a bunch of teachers and there’s one non-teacher, some luckless spouse, at the table with you? The teachers are busy trading secrets of the trade, and the non-teacher is glancing at his watch. For the non-teacher, it’s a long night. For DOE it’s been a long ten years.

Here are some of the things DOE needed to do, but didn’t, and while I recognize that DOE likes to be cutting edge, and everything has to be new, new, new, all the time, the fact is that the ideas teachers have been suggesting for ten years now are the ones that sooner or later, the DOE ought to try:

Focus on academic diversity in school admissions. As DOE knows, high concentrations of high need students overwhelm schools and undermine academic achievement across the school. Yet when DOE began to break up large schools, and institute a free-choice admission program, it did nothing to ensure that student populations under the new policies would be academically diverse. Quite the opposite in fact. Current admissions and accountability policies actually encourage most schools to market themselves to the “best” kids, leaving a system that is segregated in nuanced and troubling ways. And that means that other schools, like the ones that are the focus of this post, wind up with higher concentrations.

Giving families choice in a city as large as New York is a great thing and there’s no reason it shouldn’t continue. But as families select schools during the admission process, the DOE must ensure that the schools accept a diverse population in all but a few select schools that have special admission policies driven by stellar academic achievement, special talents, or truly unique programs.

Connect the schools. High-concentration schools need to be brought into a common fold where they can share what they know, receive direct on-the-ground support, and be directly connected through the central DOE to the best thinking on education. That’s what these schools need, but with DOE the attitude is basically “into the deep water with you, and let’s see if you can swim.” For the DOE, schools are start-up businesses and principals are their entrepreneurs who must choose networks that in turn spend a lot of time marketing various services and approaches to principals. Across a city of about 1600 schools, each school is expected to reinvent its own wheel, mostly through a blend of guesswork, political and budget considerations, and historical allegiances. What a ridiculously inefficient use of talent that ought to be directed at implementing proven ideas in thoughtful ways. Schools are not Goolge, they are not Apple, they are not Starbucks. The go-go entrepreneurial philosophy that experiments with groundbreaking products, discards some and sells others — this does not work for most schools, and it certainly cannot work for these.

Focus on school-wide issues. Politically, at least, it is much easier to imply that the fate of schools rests entirely on its classrooms where teachers can be bashed and teacher evaluation can be fetishized to kingdom come. But it is a lot smarter to put some of that energy into improving the things that lift all boats. These include intelligent programming; thoughtful attendance protocols; appropriate student academic and emotional support; and a school-wide shared understanding about what to do when students in crisis disrupt the education of other students. Schools that focus on these things do better academically, but very little of the DOE’s talent goes to figuring this out and scaling it up. All minds are bent only on how to leverage the results of Progress Reports and find someone to blame.

Or someone to incentivize. Because in response to the lack of improvement for high need students, the DOE ‘s only solution so far is giving schools extra credit if they move self-contained students out of their small (expensive) classes. Shutting their schools didn’t work; it’s hard to see how it will help to shut their classes.

Academic diversity, a single network, and a focus on school-wide conditions — those are just three suggestions, and I know they are suggestions that are likely to be ignored. But as we head into the closure season, let’s not be fooled into thinking that we are somehow improving schools. When new schools close at the same rate as the old ones, we know that it’s a bankrupt strategy, and it’s time for DOE to know that too.

1 The new high schools included in this post are those opened in 2003 or later, and established long enough to receive a Progress Report grade. Data can be found in the Progress Report data sets here.

Unfortunately the system doesn’t work the way it was intended. The UFT’s Jackie Bennett first reported on this in early 2010. She found that high schools with high percentages of high need students (special education, ELL, overage for grade on entry) were consistently scored and graded lower than schools that didn’t have such students (here and here). That year the DOE announced it was changing its peer index calculation to better account for the student characteristics that could influence results. At that time, we were hopeful, but not optimistic, that progress report card grading would improve.

To determine whether our pessimism was justified, I subjected the progress report card performance, progress and overall scores for each of the past three years to a correlation analysis. The 2009 correlations were my base year or barometer for the level that caused the DOE to revise its peer index calculation. The 2010 correlations were a measure of whether the DOE’s adjustments were effective in removing the influence of student characteristics in the data. The correlations for this year’s data were run to show the degree to which any relationship might still exist. I found that the DOE’s peer index adjustments moderately reduced the bias in the report card scoring for 2010 but that in this year’s results the association returned and is close to or exceeds the 2009 levels that warranted adjustment. The table below shows the correlation results.

For 2009, the performance, progress and overall scores show a moderate, negative correlation with a school’s percentage of special education students (a correlation of .3 to .49 is typically considered moderate)². The negative sign indicates that as the percentage of special education students rise progress report card performance, progress and overall scores declines; a pattern that favors schools with lower levels of special education students. The 2009 correlations also show a negative relationship between the progress report card metrics and the percentage of overage students, but this relationship is weak (a correlation of .1 to .29 is typically considered weak)³. Last, the table shows a positive and moderate correlation between the progress report card measures and eighth grade Math and ELA suggesting that the progress reports favor schools that accept students with higher standardized test scores.

For 2010, the table shows that most of the correlations declined in strength from the 2009 level. In many cases the correlations reached a level indicative of a weak relationship between the student characteristic and progress report card measure. While there are two correlations that remain troublesome, the one between eighth grade standardized test scores and performance and the one between self contained special ed and overall score, it appears that the DOE peer index adjustments reduced the effect that student characteristics were having on progress report card scores and grades.

With this year’s data however, the correlation coefficients rise for every student characteristic and progress report card measure with the exception of percentage of overage students. The correlations rise so much that the one between performance and eighth grade scores approaches a level that is typically considered strong (a correlation of .5 or higher is generally considered large or strong)⁴ and the ones between progress score and percentage of special education and percentage of self-contained move back to the moderate level.

What does all this mean for the NYC education community? It means that the DOE’s methodology of sorting schools into groups of “peer schools” is not working across the board. It means that for some schools as the percentage of special education and/or self-contained special education students rise the school’s progress report card score and grade will decline as this year’s results show.

¹ Meyers, Lawrence S., Gamst, Glenn, Guarino, A. J. (2006). Applied Multivariate Research Design and Interpretation; Sage Publications; Thousand Oaks, CA.
² Ibid.
³ Ibid.
⁴ Ibid.

The particulars are a little technical, but the impact is not. The DOE uses the Blue Book to decide on co-locations. It is also used to assign students to a building, add grades, bring in special education programs, and determine the multi-billion-dollar capital spending plan.

But in 23 percent of school rooms that auditors checked on, the Blue Book either gave the wrong size or the wrong function. For example, the room was described as a resource room but was really being used as an office, or the room was reportedly big enough for 28 kids when actually it could only hold 20.

Principals are supposed to collect and report the data, but the auditors concluded that they really don’t know how to do it or how the information is used. Worse, the auditors said, even in schools that the School Construction Authority (the book’s authors) had specially reviewed last year to verify the principals’ findings, the room functions and capacities were still wrong.  And curiously, the net result of the errors is that SCA tends to underestimate how much overcrowding is going on in schools.

Parents, politicians and teachers, who know exactly how crowded a particular school building is, have been citing problems with the Blue Book for a couple of years now. (UFT researchers have literally spent weeks trying to square data in the book with the reality on the ground.) The DOE’s response, in a letter appended to the report is not encouraging: the audit emphasizes “technical reporting errors that have no impact whatsoever on capacity and utilization calculations,” they write.

Really.