The Perils Of The Snapshot

With characteristically misleading hyperbole, today’s New York Sun proclaims that “Math Scores Show Charters ‘Working’.”

A snapshot comparison of the recently announced Math scores on the state exams shows that 74% of New York City charter school students scored proficient [3 or 4], it tells us, while 65% of New York City DOE students scored proficient.

This is not a sound educational judgment, but pure spin.

Snapshot comparisons of this sort tell us next to nothing about the comparative academic performance of the two classes of schools. Vital information is missing, such as student growth over time and the demographic profile of the classes of schools being compared. What percentages of the two classes are English Language Learners? Special Education students? Immigrant students? Students living in poverty? Without that information, there is no way of knowing whether or not it is apples and apples that are being compared.

If you want to know just how misleading a snapshot comparison of schools can be, consider the following snapshot that never made it into the pages of the Sun: high school graduation rates. Take a look at Slide 21 of the State Education Department’s April Power Point presentation on the four year graduation rates for the student cohort which started high school in 2001-02.

Now, if one were to do a snapshot comparison with the data from the table on this slide, the academic performance of New York charter schools would be nothing short of disastrous. Barely 1 in 4 New York charter school students [26%] graduated high school on time, well below the graduation rate not only of district schools as a whole [73%], but also New York City DOE schools [56%] and large city district schools generally [56%]. But we know nothing about the students of the schools in the charter school class, and how they compared to the students in the district schools — other than their graduation rates. So if we are intellectually honest, we will say that we lack the information to reach an intelligent, defensible conclusion.

That is exactly what the Sun should have said about the Math scores, but did not.

 Print