And the real number is . . . ?

The DOE took the opportunity to lash out against the UFT’s recent class size report [PDF] yesterday, telling the Daily News, “The UFT is able to reach its predetermined conclusions only by ignoring key data, such as the impact of team teaching and innovative scheduling.”

For the record, Collaborative Team Teaching classes were included in the analysis of all K-8 classes. And the “innovative scheduling” we ignored–presumably programming middle school students for reduced class sizes in some subjects–is not how DOE itself counts middle school class sizes.

Leaving aside the “predetermined conclusions” dig, this might be shrugged off as semi-harmless PR except it came in the middle of the dispute over the Absent Teacher Reserve, and is emblematic of an almost desperate effort by the department to have the data its way–only. It’s hard to trust their numbers, and that is truly alarming. Because while this is public data on public schools, in many ways the DOE owns it.

First, they have sole access to the databases that record school-level data such as teacher assignments, student performance, and spending. And second, they have the money (well, the taxpayer dollars) to pay for the data-crunching computer systems and legions of analysts. What they typically do with this unique data capacity is produce Power Points that highlight supportive findings and ignore data that might confuse or cloud the picture.

When we at UFT run numbers on the schools, with the data we can access, we are extremely cautious. Here’s why.

1. It’s like counting the stars. NYC is such a giant school district that pinpoint accuracy is very difficult. It’s a moving target too. For example, personnel counts can vary several hundred or even more from one month to the next.

2. If you give people a choice of ten categories to fit into they will choose the eleventh. Neither individuals nor schools seem to conform. For example, teachers teach 5 periods a day or 8, except the ones that teach 6 or 7, but that’s only on Tuesdays and Thursdays in another school, at least for the fall semester . . . you get the idea. The same holds true of schools. They are either K-5, 6-8 or 9-12, plus some that are K-8 or 6-12. Except for the ones that are preK-2, 5-8, 9th and 10th only, etc. etc. Counting them often requires averages, best guesses and exclusions.

3. People of good will routinely misreport, misrepresent or misunderstand the data they think they know. This happens all the time.

In sum, the “hard data” on education that so many of us demand as a condition of reform are in fact dense, “dirty” (this is a data term, meaning there are lots of errors in it) and ambiguous. So it is essential that when we count (and use the results to flog teachers or the system), we at least say what and how we are counting. Good education data analysts are humble folk who use lots of footnotes.

Now we find (using our shoe leather because we don’t have access to the teacher assigment data) that at least one third of educators in the Absent Teacher Reserve (ATR) are actually teaching full-time programs. Before the DOE went public with a claim that supposedly idle ATRs are costing taxpayers $81 million, they had the obligation to ask that question of the data. Even if the authors of the ATR report, The New Teacher Project, didn’t think to ask, DOE should have. That they didn’t calls into question that ATR finding.

Then that reponse to the class size report–wouldn’t you expect DOE to promise to do a better job next year, or at a minimum ask the UFT how it did its calculations? Making a false claim instead, impugning the results, calls into question whether as a public agency the DOE’s use of data needs some serious fixing, or independent oversight at least.

 Print