Log in  |  Search

Bundling Accountability

[Editor’s note: Sherman Dorn is the author of Accountability Frankenstein, the editor of Education Policy Analysis Archives, and an associate professor of education at the University of South Florida. He blogs on education policy at shermandorn.com.]

The controversy over giving letter grades to New York City schools last month demonstrates two features of modern accountability: it has bundled different types of accountability together, and the bundling of accountability has gone too far.

On November 5, New York City revealed single letter grades assigned to each public school in the five boroughs. Patterned after Florida’s assignment of letter grades, it boiled down statistical data to a single judgment. Unlike in Florida, the inconsistencies and illogic in the letter grades have become the center of vigorous debate.

But the controversy has obscured an important point: while the assignment of letter grades is politically clever—if students receive letter grades, why can’t schools?—it also bundles several different features of accountability into a single package. The same Joel Klein who fought Microsoft’s bundling of software is now engaged in bundling of accountability for public relations purposes.

The single letter grades allowed Schools Chancellor Klein to bundle together several different concepts—independent evaluation of schools, statistical measurement of achievement, and the reduction of all information down to a simple message for public relations purposes: the letter grade.

Of those three concepts, the independent evaluation of schools is the most valuable part of accountability. Citizens deserve to have independent information about what schools are doing. Unlike decades ago, when administrators had substantial autonomy and independence from political processes, we do not trust schools to operate without some outside check. In a school system with a million students, New York’s parents and taxpayers need to know that their schools are educating all students.

But should that independent evaluation consist only of statistical data? In New York City and in No Child Left Behind, accountability relies entirely on statistical formulae, with the argument that achievement data and hard statistics are the only objective measurement of schools.

But in our society, we have a variety of independent evaluation techniques, from health inspectors to fact-finding commissions and the court system. In these other systems, human judgment is rarely vacated in favor of statistics. The key feature of these systems is the exclusion of parties from decision-making. In real accountability systems, distance from self-interest is the measure of independence.

That independence does not require a brain-dead reliance on statistics. In most systems of independent judgment, statistics are a tool and not the sole way to make decisions. A judge who failed to use her or his judgment would rightly be accused of abdicating her or his responsibility to think. In truth, independent evaluation of schools can use different types of information, including student achievement data, visitation reports by teams of outsiders, and student and parent feedback.

The bundling of statistics with independent judgment is common in school accountability, with statistical formulae used in almost all states before 2002 and now nationwide with No Child Left Behind. But what New York City shares uniquely with Florida is the reduction of all information about a school into the letter grade, a single datum for judging schools and shifting responsibility.

I understand the appeal of this reduction. As a student, I received letter grades. Why can’t schools receive them as well? In his first year as Florida’s governor, Jeb Bush used this parallel brilliantly. In 1999, he pushed a high-stakes accountability system through the Florida legislature, promising that the transparency of letter grades would provoke schools to improve.

To many, that improvement seemed plausible. Because former Governor Bush relied on a testing system that already existed, and because letter grades are familiar to adults from their own experiences, the imposition of a giant new apparatus seemed a reasonable shift in accountability.

The new system of letter grades provoked change in Florida, but it is not clear whether they improved schools. NAEP data on reading achievement of fourth graders have improved, but Florida’s NAEP eighth-grade reading scores have stagnated, and Florida’s NAEP eighth-grade math scores are on the same trajectory that started in the early 1990s.

What Governor Bush’s accountability system accomplished was as much symbolic as substantive. For public-relations purposes, the assignment of a single letter grade to schools shifted responsibility from the state to the principal. Grades go up, or they go down, and they can also go up on a school’s streetside marquis: “We are an A School.” School grades provide a way to distribute a message about accountability in a single question: “What’s your grade?” The state of Florida has never graded itself, though Florida’s constitution lays the responsibility for providing an adequate education on the state.

The bundling of accountability is not just a way to manage schools but a way to package accountability for public relations purposes. Bundling accountability controls the message and shifts responsibility. By monopolizing judgment of schools in a centralized system, Chancellor Klein is attempting the same control in New York City.

In the same way that Microsoft bundled software to exclude competitors, Klein has bundled accountability to exclude other ways of judging schools. To the public relations honchos that Klein hired, message control is the most important feature of accountability. That is why New York City’s schools spent taxpayer money gathering data with which to smear critic Diane Ravitch. The same man who fought Microsoft on legal and moral grounds is now using tactics that he criticized when Microsoft engaged in them, and the irony is palpable and largely ignored.

What New York City’s parents and other citizens are now realizing is that bundling accountability is also bungling accountability. The letter grades are inconsistent with the judgments that parents and others familiar with schools know. The logic of so-called peer comparisons is unraveling with closer scrutiny, and some reports are emerging that the letter grades were fiddled with for political purposes.

We all can learn from the mistakes of New York City and Florida. A monopoly of centralized accountability serves the public-relations interests of the politicians at the head of those systems, not the public interest of citizens. We need to unbundle accountability, disentangling the value of independent judgment from the public relations role of a single label.



  • 1 Sherman Dorn: Guesting on Edwize!
    · Dec 21, 2007 at 2:24 pm

    […] gone and committed guest blogging over at the UFT blog Edwize. The gist of the argument is that Joel Klein’s pulling a Microsoft-like […]

  • 2 woodlass
    · Dec 23, 2007 at 3:52 pm

    There are many obfuscations with the letter grades, not only bundling – I’ve been questioning them for weeks in various places and compiled these recently into an article called “Accountability – a mantra for spinnmeisters.”
    This practice does nothing for kids or parents.

  • 3 eduwonkette
    · Jan 14, 2008 at 7:36 pm

    […] gaggle of guest bloggers to comment on the NYC Progress Reports – check out Sherman Dorn’s post on “Bundling Accountability,” Seth Pearce’s post on “The Importance of the School Progress Debate,” and my post, “The NYC […]