Log in  |  Search

Multiple Choice: A High Stakes Study Of Charter School Performance

The Center for Research on Education Outcomes [CREDO] at Stanford University today published a report, Multiple Choice: Charter School Performance in 16 States, which will create a bit of a stir in educational circles.

Multiple Choice sets a new benchmark for national research studying the academic performance of charter schools and district schools: it compares “virtual twins” [charter and district students with the same demographics]; it employs a significant NAEP data set; it looks at longitudinal growth; and it covers a broad cross-section  of states, including many which had not been included in previous studies .  Moreover, CREDO is a think tank which has collaborated with national charter organizations such as the National Alliance for Public Charter Schools and the National Association of Charter School Authorizers in the past [see this study, for example], and Multiple Choice was financed in part by the Walton [Wal-Mart] Foundation, so its findings can not be easily dismissed as the work of charter school critics. [Although none of this stopped an effort by Nelson Smith of the National Alliance for Public Charter Schools to spin away its most important conclusions.]

Here are the reports’ main conclusions, word for word, a set of findings which the authors describe as “sobering” [p. 6]:

  • Charter school students on average see a decrease in their academic growth in reading of .01 standard deviations compared to their traditional school peers. In math, their learning lags by .03 standard deviations on average. While the magnitude of these effects is small, they are both statistically significant.
  • The effects for charter school students are consistent across the spectrum of starting positions. In reading, charter school learning gains are smaller for all students but those whose starting scores are in the lowest or highest deciles. For math, the effect is consistent across the entire range.
  • Charter students in elementary and middle school grades have significantly higher rates of learning than their peers in traditional public schools, but students in charter high schools and charter multi-level schools have significantly worse results.
  • Charter schools have different impacts on students based on their family backgrounds. For Blacks and Hispanics, their learning gains are significantly worse than that of their traditional school twins. However, charter schools are found to have better academic growth results for students in poverty. English Language Learners realize significantly better learning gains in charter schools. Students in Special Education programs have about the same outcomes.
  • Students do better in charter schools over time. First year charter students on average experience a decline in learning, which may reflect a combination of mobility effects and the experience of a charter school in its early years. Second and third years in charter schools see a significant reversal to positive gains.

These patterns vary considerably by state.

States With Lower Average Academic Performance By Charters States With Higher Average Academic Performance By Charters States With Academic Performance By Charters Which Is Mixed Or Not Different



New Mexico




Colorado [limited to Denver]

Illinois [limited to Chicago]




District of Columbia


North Carolina

Looking at all 2403 charter schools studied in Multiple Choice, the authors found that 17% had a “positive and significant effect” on student achievement gains, 37% had a “negative and significant effect” on student achievement gains, and 46% had a “no significant effect” on student gains. [p. 44]

There is one counter-intuitive finding in the report that is not convincing to this reader. The report finds that “the presence of charter caps puts significant downward pressure on student results.” [p. 40] Yet the states which are best known for having either no cap or no meaningful limit on the numbers of charter schools and little or no regulation fall either into the under-performing [Arizona, Ohio and Texas] or  no significant effect [California] categories. It is not at all clear from the report how the various factors were weighted to produce such a result.

This last point is important, especially in the current political climate. If one should carry from this report any lesson, it has to be that the simple creation of new charter schools does not improve educational performance: they are clearly no magic bullet to what ails American public education. Rather than create a system of unlimited mass production of charter schools, in which their sheer quantity becomes the prime directive, it is essential to develop a system to craft quality charter schools which enhance public education. It is hard to see how that is happening on scale in the “wild, wild west” of Arizona charter schools, for example. Charter school advocates who ignore the findings of this report and turn away from its imperative to focus on quality charter schooling do so at the peril of the movement’s long-term health.



  • 1 Gideon
    · Jun 16, 2009 at 9:02 am

    I think the last bullet point is one of the most important. Given time, charter schools do raise student achievement, so we should be looking at charter schools that have been open three or more years when we try to evaluate their effectiveness.

  • 2 Kyle
    · Jun 24, 2009 at 9:36 am

    I think the last paragraph is the most important. Quality of charter schools is the key, not Quantity. Parents need to be smart consumers of education as with anything else, and choose high quality Charter Schools. That said, you can have neither quality nor quantity if your state continues to prohibit Charter Schools. This is the situation in Maine, and it is very frustrating.

  • 3 Tabithahargrove
    · Jun 25, 2009 at 10:15 am

    Edu-Blog Paper

    How can one compare a public school with a charter school? This has been of much debate at my school and other schools in New York City. As I close out my first year of teaching at a Middle School in District 8 I feel the gains we have made from getting off the SURR list, receiving an implementation grant ($250,000), and increasing our ELA and Math State scores from 45% TO 75% of the students with 3’s and 4’s goes unnoticed in the realm of charter schools. I feel many of us have to work twice as hard to prove that public schools are good schools and that public school teachers can perform and help the students grow just as well as the charter school teachers. As I began to read the Edu-blog on charter school funding and public school funding I began to quickly realize who was favored.
    The debate of how equal the school funding is for charter and district schools has never ceased to amaze me. While the charter schools are screaming for more money because they feel they are underfunded, the district schools are seeing more of their money being taken away and believe the charter schools are overfunded. This quote from a charter school blog on edwize.org states, “the fact that charter schools receive more than $12,000 per pupil, compared to an average of about $8000 per gened student that is sent to school level for traditional public schools, might lead one to conclude that charter schools actually receive a higher level of public support than traditional public schools.” [Haimson, Leonie, 2009] This is a significant difference from traditional public schools and I am still confused why this is? In this blog it has been said many times that the charter schools need more money because they need to pay for the building they work in and the increased teachers’ salary. This blogger then also lists many of the supports from the state that the charter schools receive for free such as facilities, energy, food, and transportation per pupil funding, so again why are the charter schools receiving more money per pupil as well as receiving free amenities if now the money that they would have needed for these amenities are now free?
    As the next fiscal year begins all of the funding for schools are frozen which means all schools will feel a hit but more so the traditional public schools. In my school we lost a little over $500,000, which meant 5 teachers, 6 school aides, and 2 social workers had to be accessed. This has confused me even though I am only a first year teacher, I can’t understand why a school that has made so many gains is being cut of funding. Everyone who is in the school contributed to our gain as a school and now they need to find new jobs but the charter schools who do not need an increased teachers salary or extra money gets to keep their frozen accounts and the only thing that takes the hit are the extra programs (all this was said in the blog).
    In conclusion I feel this battle of equity among charter and traditional public schools will be ongoing. Both sides will always want more money and always find faults in others who receive that money. Based on numbers alone it shows that traditional public schools have more students therefore they should receive more money. In charter schools having smaller classrooms 20:1 should not be compared to 35:1 in a traditional public school setting, should not base the opinion that charter schools are better. They are allowed to spend more individual time with students improving their overall state scores. If the traditional public schools were allotted more money then our class sizes would change as well and our overall state scores would increase and we would be able to be considered equal to charter schools.

  • 4 Shabad Siu
    · Jun 26, 2009 at 4:59 pm

    Shabad Siu
    Edu-Blog Entry
    Funding for charter schools versus traditional schools has always been a hot topic. As a first year teacher teaching in a traditional school setting I have heard a lot of negative arguments against funding for charter schools. Going straight from college to a traditional school I did not get a chance to explore charter schools. I always wondered why there was a lot of buzz about charter school. When I went on interviews the majority of the recruiters were from charter schools and they displayed their schools to be a glorious place. I have realized otherwise after completing my first year as a teacher. According to Mandate for Change, Charter schools are one aspect of the five-part cure for fixing public school education.
    Charter schools are innovative public schools that are accountable for student results by delivering tailored educational services based on the needs of the communities they serve. They are one of the most successful educational reforms of our country and they are growing fast. Charter schools are judged based on student achievement. Their leaders also can focus mainly on setting high academic goals for their students rather than worrying about procedures that gives other administrators a run around. Also, these schools have specialized programs meeting students needs in subjects such as reading, writing, and other arts. Some also contain programs like dropout prevention and adult education. So they seem to be the answer to bridging the educational gap that we are facing today. But that is not the case; I have seen other teachers that are in charter schools facing the same discipline problems we have in a traditional school setting and also dealing with the same amount of paperwork. Our school has tremendously improved the ELA and Math scores this year, yet our teacher are being excessed due to budget cuts. According to the blog, charter schools that do not need to raise teacher’s salary or need any extra money are able to keep their accounts frozen and if they do need to cut something they hit the extra programs. But, teachers in a traditional public school setting are losing their positions because of budget cuts. I think this is unfair and with fewer teachers there will eventually be more discipline issues which will affect the scores.
    According to a high stakes study of charter school performance, these schools have better academic growth results for students in poverty. Also, charter schools have different impact on students depending on their family backgrounds. They also mentioned that Blacks and Hispanics have significantly worse learning gains as compared to traditional school setting. This can be due to excessive pressure placed on the students. I am not sure if there is an effective way of measuring the effects of charter schools versus traditional schools because so many factors affect student’s performance. Even though they kept the demographics the same what takes place in the student’s personal lives effects their concentration and dedication towards learning. I sincerely believe that the traditional school system should receive as much support as traditional schools since the performance differences are minute.
    Word Count 519

  • 5 Shawlatan
    · Jun 27, 2009 at 1:03 am

    The study is meaningless when it comes to allowing children to self-actualize and become good world citizens. It is meaningless because it tests students the way traditional schools do. Traditional school’s test results show how well kids take tests and how well they regurgitate what they have swallowed whole by rote. Tests don’t measure understanding; that is, how well a kid has mastered a subject (of her choosing).
    Also, most charter schools deviate little from the traditional school model. They are different in degree but not in kind.
    Education must be changed fundementally for there to be thinking, questioning, self-motivated, humans who will better understand themselves and the world around them.
    I recommend that Summerhill, Sudbury Valley, and Albany Free School models be emulated around the world. Because traditional schools (and most charter schools) are as Howard Zinn said, totalitarian. I say school is not free inquiry; it is indoctrination. Schools today quell curiosity, and build good little followers and suck ups and bad critical thinkers.
    Wake up teachers, and get a voice. Read John Taylor Gatto’s, The Underground History of American Education.