Log in  |  Search

Is There Real Evidence for Lifting the Start-up Charter School Cap in New York State?

In a New York Daily News op-ed this week, the head of the NYC Department of Education’s charter school arm, Paula Gavin, makes what is now a familiar argument from the DOE and a number of charter school advocates here in New York. Charter schools in New York City are outperforming the regular public schools on the standardized state tests, she tells us, but the cap on the number of charter schools written into the state legislation authorizing their establishment prevents the formation of more of these quality schools. Ergo, the cap must be removed.

This same argument has been made with some regularity since last spring, when the number of start-up charter schools began to approach the cap of 100 and the results from the state tests were made public. See here and here and here.

Now call me a skeptic if you want, but I have learned over time that it is best, in the words of Ronald Reagan, to “trust and verify” when claims are made by agents of the NYC Department of Education. So I decided to take what the DOE and the other charter school advocates put forward as their strongest case, the eighth grade English Language Arts exam results, and see if their claims could be sustained.

Using the New York State Education Department’s report of exam results by school, I prepared a chart for the test results on the eighth grade ELA exam for all of the charter schools in New York State. This chart is reproduced below [at the end of this post]. For the non-NYers, exam results in NY State are grouped into four general categories: a grade of 4 indicates the student is exceeding the state standards, a grade of 3 indicates that the student is meeting the standards, a grade of 2 indicates that the student is approaching the standards, and a grade of 1 means that the student is well below the standards. Schools can be easily compared by the numbers of their students that fall into these categories — specifically, by the percentages of their students meeting [Level 3] and exceeding [Level 4] standards.

The results were quite instructive. First, it became clear that the numbers were being spinned when the terms of comparison were made only between New York City charter schools [the first six on the chart] and New York City district schools, and not between all of the charter schools and district schools in the state which had taken the exam. [An immediate clue that something was amiss came in the fact that the remedy being proposed -- lifting the cap -- would be for the entire state, but the comparisons being offered were only for New York City.] When all of the schools were examined on a state-wide basis, only 4 of the 15 charter schools giving the exam had posted results better than the state-wide average for district schools, with many charter schools falling far below that benchmark. Since 3 of the 4 schools scoring above the average were from New York City, limiting the terms of comparison to NYC schools alone made it possible to present a set of results on which charter schools had fared poorly into a set of results on which they were doing well. [It is also worth pointing out that the statewide average is not a very high benchmark: one could have a little more than 1/2 of one's students score in Levels 3 and 4, and beat the statewide average.]

It is important here to understand that in New York State, the two main chartering agencies — the Regents of the New York State Department of Education and the Trustees of the State University of New York, have statewide jurisdictions, and authorize charters all over the state. Both Bronx Preparatory and Harbor Science and Arts were chartered by state agencies that have chartered other schools across the state. [The local school district may charter schools with the approval of the Regents, but this has been rare to date outside of New York City.] Thus, if we want to have an accurate forecast of what types of schools would be chartered if the cap was lifted, we would need to look at all of the schools which had been chartered in the state. To arbitrarily limit the terms of comparison to New York City is to remove the great bulk of the low performing charter schools from the examination.

Secondly, it is interesting to look more closely at the charter schools in New York City. Of the six charter schools in the city, four were conversion charter schools — that is, schools which had been in existence as part of the school district for some time before becoming a charter school. Under NYS charter school law, any existing district school can, through a procedure involving such checks and balances as a democratic vote of the parents, opt to become a charter school. Conversion charter schools tend to be schools which are doing well, but chafing at the heavy and truculent hand of the district  management. Three of the higher performing schools in New York City — KIPP, Renaissance and Beginning with Children — were conversion charter schools; only one conversion charter school — Wildcat Academy — did poorly. The number of students taking the exam at Wildcat was rather small, the lowest in the state, leaving some question of whether the sample accurately reflected the school.

This is significant for two reasons. First, under the state law, there is no cap on conversion charter schools. There are no limits to the number of KIPP or Renaissance type conversion charter schools the DOE could charter under the state law. The cap in the state law applies to start-up charter schools, where the record of performance vis-à-vis district schools was quite poor. This means that the charter schools whose performance is being used to argue for lifting the cap on start-up charter schools are not even start-up charter schools themselves.

Secondly, it is no secret here in New York City that the reason why the DOE has been so active in its campaign to lift the cap on start-up charter schools is that Chancellor Klein sees this as a way to create an entire cohort of public schools outside of the UFT’s collective bargaining agreement. It is his contention that the collective bargaining agreement stands in the way of good education. But one of the salient features of a NY State conversion charter school is that they retain any pre-existing collective bargaining agreement when they convert. KIPP, Renaissance, Beginning with Children: they all operate under the same collective bargaining agreement as NYC district public schools. Thus, it is the charter schools operating under the UFT collective bargaining agreement that are the highest performing charter schools. The reality here is at direct cross purposes with the intentions of the DOE under Klein, and with its rhetoric about the negative effects of union work rules. From this sample, it would appear that such organizational features as contractual limits on the size of a class or the number of classes a teacher can be assigned to teach in a row actually improve education.

Now, to be fair to the charter schools across New York State, one must note that, for the most part, these are new schools, and that it takes a few years for a new school, charter or otherwise, to get to full speed. Some of those schools with poor performances on the state exam will improve with time. Others will not, and they should be closed down, just as low performing district schools are closed down. To the credit of the SUNY Board of Trustees, they have revoked the charters of two of their lowest performing schools. Charter schools should also receive the same funding as public schools, especially now that even Checker Finn has come to understand that full funding of public schools is a necessary [albeit not sufficient] condition for high levels of academic performance. [It is remarkable what insight can come on the road to Damascus, when one gets knocked off a charter school one is riding.]

But taken as a whole, the performance of New York charter schools on the eighth grade ELA exam tells us that charter school advocates need to be focusing on significantly improving the quality of the education in New York charter schools, as well as the performance of their students on state exams, before they can make a honest argument that this performance supports the lifting of the cap on the number of start-up charter schools.

I write this as a supporter of charter schools, and as someone who was deeply involved in and supportive of the UFT application for its own secondary charter school. I want charter schools in New York State to succeed, charter schools of the sort that Al Shanker envisioned when he first promoted the idea — schools that grow organically out of groups of educators and local communities organizations dedicated to the students they teach, and not out to make a quick profit off of public education. But success in education requires honesty about the strengths and weaknesses of the ventures in which one is involved. Less than forthright spins of the data of the sort that is being done here invariably come back to haunt an effort, no matter how well-intentioned they may be.

It is also time, I would argue, to take a critical look at the theory of how to produce quality schools which has been in operation at the DOE under Klein. That theory, in sum, is that quantity produces quality. That is why the DOE has been starting up incredible numbers of new small schools every year — well over 100 one recent year — when it is clearly outstripping its capacity to ensure that every new school is a quality school. While new schools clearly need, as a minimal condition for success, an experienced school leader with a collaborative method of leadership and a staff with a solid corps of experienced, accomplished teachers, far too many new schools in New York City are being started with inexperienced leaders and entirely novice staffs. Moreover, with so many new schools starting at once, administrative regions are clearly not able to provide the supports that those new schools need. But the theory at the DOE seems to be that if you just keep producing more and more of the new schools, quality schools will emerge. It now appears that the DOE would apply the same theory to charter schools, on the assumption that creating many more charter schools is going to make charter schools better.

This theory takes me back to the halcyon days of graduate school, when one had the time to read books of a purely historical value. One such assignment, which I certainly hope has faded from required reading lists in the intervening years, was Engels’ Dialectics of Nature. Engels argued that the theory of evolution demonstrated the truth of the dialectic, and vice versa, because the accumulation of quantitative adaptive changes in nature produced a qualitative transformation — evolutionary advance. The problem was that Engels had a less than complete grasp of both science and philosophy. The theory of evolution Engels was espousing was Lamarckian [which could not survive modern genetics], not Darwinian, and Hegel would have gagged on Engels’ version of the dialectic. Quality simply does not appear magically out of quantity, in nature or in education — the DOE’s theory of small schools and charter schools to the contrary. One has to build quality schools, painstakingly and carefully, piece by piece. Teaching and learning are incredibly complex activities, and the institutions that are capable of supporting  quality teaching and learning are no less complex. As Rafe Esquith, the teacher who provided the original inspiration for KIPP never tires of telling his students, “there are no shortcuts.” It would appear that the DOE has yet to learn that basic lesson.

EIGHTH GRADE ELA EXAM RESULTS FOR NY STATE CHARTER SCHOOLS
Charter
School
#
Students
Tested
Mean
ELA
Score
Level
One:
Level
Two:
Level
Three:
Level
Four:
Above
NYC
Average
Above
NY State
Average
Lindsay
Wildcat
Academy
12 679 #: 0
0%
#:11
91.7%
#: 1
8.3%
#: 0
0%
NO NO
Harbor
Science
& Arts
27 689 #: 1
3.7%
#: 17
63%
#: 9
33.3%
#: 0
0%
YES NO
Kipp
Academy
49 711 #: 0
0%
#: 14
28.6%
#: 30
57.7%
#: 5
10.2%
YES YES
Bronx
Preparatory
52 699 #:1
1.9%
#: 21
40.4%
#: 30
57.7%
#: 0
0%
YES YES
Beginning
with
Children
41 697 #: 0
0%
#: 26
63.4%
#: 14
34.4%
#: 1
2.4%
YES NO
Renaissance 50 700 #: 0
0%
#: 24
48%
#: 24
48%
#: 8
3.5%
YES YES
Totals
for NYC
Charter
Schools
231 #: 2
0.9%
#: 113
48.9%
#: 108
46.8%
#: 8
3.5%
5 OF 6 3 OF 6
South
Buffalo
49 696 #: 0
0%
#: 28
51.7%
#: 17
34.7%
#: 4
8.2%
NO
Stepping
Stone
45 680 #: 5
11.1%
#: 31
68.9%
#: 9
2%
#: 0
0%
NO
Enterprise 49 676 #: 10
20.4%
#: 31
68.9%
#: 8
16.3%
#: 0
0%
NO
Buffalo
Academy
for
Sciences
66 674 #: 9
13.6%
#: 48
72.7%
#: 9
13.6%
#: 0
0%
NO
Westminster 53 699 #: 0
0%
#: 26
49.1%
#: 27
47.2%
2
3.8%
YES
School
for
Applied
Technology
110 685 #: 8
7.3%
#: 72
65.5%
#: 30
27.3%
#: 0
0%
NO
Rochester
Leadership
48 689 #: 1
2.1%
#: 33
68.8%
#: 12
29.2%
#: 0
0%
NO
School
of Science
and
Technology
102 679 #: 13
12.7%
#: 73
71.6%
#: 15
14.7%
#: 1
1%
NO
Syracuse
Academy
83 687 #: 8
9.6%
#: 44
53%
#: 29
34.9%
#: 2
2.4%
NO
Totals for
Non-NYC
Charter
Schools
605 #: 54
8.9%
#: 386
63.8%
#:156
25.8%
#: 9
.1%
1 OF 9
Print

8 Comments:

  • 1 curious2
    · Aug 25, 2005 at 3:26 pm

    Hey Leo,

    What do you think about the choice argument for lifting the cap, i.e. parents seem to want to send their kids to charter schools so let’s create more? Do you think the parents that are going through the trouble to send their kids to charter schools are making a mistake? Are they sending their kids to inferior schools from which they had previously attended?

  • 2 msd2005
    · Aug 25, 2005 at 10:17 pm

    Comparing average charter school performance to the state average is garbage statistics. It’s apples and oranges. A valid comparison would compare charter schools to other public schools with similar demographic characteristics. Urbanicity, race, poverty, SpEd and ELL are important factors in such a comparison. Moreover, these snapshot comparisons don’t tell as much as comparing average gains over time which one of these days we should have the capacity to measure when everyone takes the state test in grades K-12.

  • 3 Leo Casey
    · Aug 28, 2005 at 9:26 am

    Curious2:
    There is a difference between marketing a school and producing a quality school. It is significant that, when the SUNY Board of Trustees decided that they had to shut down two of the schools they had chartered for low performance, there was resistance from the parents at one of those schools here in NYC. The school had been marketed well, but not made into a quality institution.

    Msd2005:
    You are correct that comparisons of schools should be done with other schools serving a demograpgically similar student body, and that comparisons should be over time [value-added]. If the DOE and other charter school advocates were serious about the claims they had made, they would have done so. They did not. Note that I simply pointed out that the claims they made could not be sustained.

    Last spring, I attended the NY State Charter School Association covention where such an analysis was provided. The presenter corss-referenced the percentage of free lunch of students [as a proxy for poverty] with the performance on the state exams for all of the schools in the state, plotting them on a graph. He then threw up a line of regression, which showed that as a general rule, the higher the concentration of poverty the lower the test scores, and vice versa. He then put the charter schools on the graph — and the great bulk of them fell below the line of regression. His message, entirely correct: charter schools in NY State have a lot of work to do to make their schools into quality schools.

  • 4 curious2
    · Aug 31, 2005 at 11:01 am

    Hey Leo,

    Thanks for your reply.

    Your reply raises a potentially minor issue in response to very direct and basic questions. You think parents shouldn’t have choice because they might be mislead by marketing? Should this also apply to food, clothing, toys, airfare, cars, etc? These are all important things to children as well. Seems like a flimsy argument to me.

    Also, I think msd2005 pointed out a basic flaw in your lengthy posting. In your reply to him you change your basic argument to one of raising doubt rather than demonstrating inferiority and then try to make the original point again by referring to an unknown presenter at some conference you once attended. Again, this seems like a flimsy argument.

    In both cases, it seems like your original posting was mostly a long flawed framework for justifying your opinions on how schools should operate. I think the argument would be stronger if you left out the weak data-based justifications and focused on your experience-based suspicions.

  • 5 Leo Casey
    · Aug 31, 2005 at 1:37 pm

    Curious2:

    Some market choices are low risk, and some market choices are high risk. The choice of a toothpaste is low risk, and so the government regulates it in minimal ways — some basic FDA regulations that ensure that the product is not a hazard to the consumer. You and I choose our toothpastes based on marketing by the brands, but it matters very little what tooth paste we choose. By contrast, The choice of a brain surgeon is high risk, and so the government regulates it in rather substantial ways — we don’t allow hospitals and doctors to perform such surgery without undergoing rigorous and ongoing forms of licensure and certification. There is no way that either you or I would know how to obtain and evaluate the specialized knowledge which would tell us that a particular doctor or a particular hospital is a safe choice for brain surgery [and both of us are well-educated people], and so we have to rely upon government to do a preliminary ‘vetting’ through regulation. We assume that if the government has certified a doctor or a hospital, minimal conditions have been met.

    As market choices go, the choice of a school is high risk in the life of a child. A bad choice can not be simply undone, or years of a child’s education simply recuperated. Moreover, there is a degree of specialized knowledge involved in assessing the performance of a school, and a non-educator, or even an educator without access to certain kinds of information, is not in a position to make a sound decision on the quality of a school all by themselves. Government thus has a special responsibility to ensure the quality of the schools it makes available to the public. These schools should meet minimum performance standards. Schools which fall below those standards should be closed down.

    The point of charter schools was to provide parents with more good choices among public schools. Insofar as charter schools in a state like NY are not yet performing at the same level as the district public schools, this policy objective is clearly not being met. When the state legislature passed the law authorizing the formation of charter schools, they placed a cap on the number of charter schools, with the idea in mind that charter schools would need to prove themselves as at least equal to the district schools before their numbers were expanded. The point of my analysis was that, contrary to the argument the DOE is making, that burden of proof has not been met.

    As to the second part of your argument, you seem to want to ignore the fact that what I did was examine an argument put forward by the DOE and other charter school advocates, and show how, on its own terms, it was wrong. In a somewhat strange twist of logic, I am to be held responsible for the shortcomings of their argument — the fact that they did not control for the demographics of the student population. But in point of fact, their failure to control for socio-economic class in the student population is just one more reason why the evidence they offer for their argument to lift the cap is not convincing. And since they are making the argument to change the law, the burden of proof is on them.

    Finally, you want to dismiss the fact that a conference of the charter schools themselves, with every incentive to present charter schools in a positive light, an analysis was made that showed when controlled for socio-economic class, charter schools were not performing at the same level as district schools. [I actually thought that the presentation spoke highly of the charter school association, because it meant that they were paying serious attention to the quality of their schools.] This presentation is quite important because it shows that the full analysis is being done, that the DOE knows full well what is in that analysis [they are present at such events], and that the reason why they choose not to make such a comparison in these arguments is because they know that it will reflect poorly on the argument they are making. You can’t have it both ways: if this sort of analysis is the most insightful, then you must recognize what it says, especially when it is coming from inside the charter school movement itself.

  • 6 Edwize » “Noble Lies?” EDDRA And The Daily News
    · Oct 3, 2005 at 1:31 pm

    [...] And that is only the beginning of the dishonesty here. A few weeks back, we took a look here at Edwize at the argument of the New York Center for Charter Excellence for the lifting of the cap on charter schools, based on the earlier reporting of the results of the ELA exams alone. [The ELA results were published a few months before the results of the Math exams.] Then, as in the more recent case of the Math exams, the charter secondary schools in New York City performed better than the charter elementary schools, so we took the stronger argument for charter school excellence, the secondary schools, and examined it in some detail. What we found there was that the separation of New York City schools from the rest of New York State – an artificial distinction, since schools are chartered on a statewide, not a city, basis – was designed to gain maximum benefit from a few charter secondary school statistical outliers in New York City, while conveniently ignoring the much greater mass of low-performing charter secondary schools throughout the state. We did a similar study of the secondary charter schools giving the 8th grade Math exam, and the results were virtually identical. If you examine the table at the end of this post, you will find that while three of the six secondary charter schools in New York City were above the state average, and five of the six in New York City were above the city average, only one of the other nine secondary charter schools in the state were above the state average. When the entire pool of secondary charter schools in New York are examined, therefore, the results were nothing approaching the superior performance claimed by the Daily News. It is essential to examine the state wide results because if the cap for charter school were lifted, it would be lifted for the entire state, and it would be lifted for the chartering agencies which chartered all of the secondary charter schools in New York State. [...]

  • 7 Edwize » NY Post: CHOICE IS GREAT — EXCEPT FOR TEACHERS WHO WANT A UNION
    · Nov 4, 2005 at 12:30 pm

    [...] It’s always entertaining to watch the mighty pens of the New York Post editorial staff twist and turn around an issue when the plain truth is too much for them to bear [and, for that matter, to bare].   The latest episode is the Post editorial ‘Zap The Cap’ which denounces – hold on to your seats, boys and girls – the UFT and Randi Weingarten for the fact that the cap on the number of charter schools in New York State remains in place.   Let us leave to the side the Post’s editorial sortie into the prose style of Dr. Seuss, as some of us always saw this as an inevitable rhetorical turn, given the limited repertoire of its barely post-adolescent editorial writers and the fact that it pitches its newspaper to a third grade reading level.     What is interesting here, as in most Post editorials, is what is not said. Specifically, the Post dances around the facts in this Daily News article, which appeared less than a week before the Post editorial. What is now on the record is the fact that Randi indicated the UFT’s willingness to consider an increase in the cap on the number of charter schools. The only condition for UFT support is that the law also be amended to ensure that teachers in charter schools who wanted to be represented by a union could do so without hindrance, by certifying a union as the collective bargaining agent once it is verified that a majority of the teachers have signed union membership cards.   But for those, both in the Post editorial offices and at Tweed, who see charter schools as a way to create non-union schools rather than schools of excellence, the idea that teachers in charter schools would have the free choice to unionize is anathema. Better that the number of charter schools be frozen than there be more unionized charter schools.   Choice, it would appear, is good for everyone but teachers who want a union.   [As devotees of the historical record, we can’t let this moment pass without also pointing out that the Post once again plays fast and loose with the facts in its editorial. It asserts that low-performing public schools are not closed down. We don’t know where they live, but in New York City over the last five years, no less than seventeen different high schools deemed to be low-performing have been closed down or are currently in the process of phasing out. And the Post misrepresents the academic performance of charter schools in New York State as superior to that of regular district schools. We have dissected that particular falsehood at some length, here and here.]  [...]

  • 8 Edwize » WHO’S AFRAID OF TEACHER VOICE? CHARTER SCHOOLS AND UNION ORGANIZING
    · Nov 17, 2005 at 11:17 pm

    [[…] You could drive a fleet of non-union trucks through that hole, which ALF and Jackson Lewis will gladly defend in court. THE NYC DEPARTMENT OF EDUCATION – CHARTER SCHOOL – ALF CONNECTION As offensive as it is, neither the Jackson Lewis pamphlet nor the Jackson Lewis presentations were unforeseen: no one expects that the leopard will change his spots. What was remarkable, however, was who appeared on the other panel at […]