Log in  |  Search

Vanishing Students, Rising Scores:
Middle School Charters Show Alarming Student Attrition Over Time

Much has been said about the high demand for charter schools in entrance lotteries, but little about the choices families make once their children are actually accepted. Looking at attrition rates is important because they may tell us about the choices parents make about their children’s schools. Attrition may also indicate that some students are encouraged to leave or have been expelled. And, if proficiency rates rise over time in schools with heavy attrition, those percentages may reflect the change in the student cohort, as well as or instead of academic progress.

I took a look at changes in the size of student ELA testing cohorts for the thirteen middle school charter s that have more than one year of data between 2006 and 2009.[1] Since all students in the testing cohort must take tests every year, I was able to determine whether cohorts shrink or grow over time.

As it turns out, high-performing charter middle schools in the New York City also have extremely high rates of attrition in their testing cohorts :

  • Eight of the thirteen schools have enough data to allow us to examine cohort size between 5th grade, when students enter, and 8th grade, when they graduate.[2] In four of these schools, more than 25% of the students vanished from the cohort. Of these four schools, three saw cohort declines of 30%, and one lost nearly 40%. All of these charters have been nationally or locally acclaimed as great schools that are in high demand. The average attrition for this group of eight is 23%. (charts follow.)
  • These attrition rates contrast starkly with what I found in regular public schools, where the size of cohorts tends to remain the same or rise. (charts follow.)
  • The remaining five of the thirteen schools are too new to have testing cohorts that span all grades. However, most have high attrition through the grades already completed. Taken as a whole, only two of the thirteen middle school charters show no shrinkage in the size of their testing cohort, thus approaching parity with public schools citywide.
  • As the testing cohorts shrink, the percent of remaining students who are proficient rises dramatically. Seven of the thirteen middle school charters have more than 20% attrition in their testing cohorts. All seven have proficiency rates that rise to over 90%, with an overall average of 94%. That rate drops 21 points for charters with less than 20% attrition.[3] (charts follow.)
  • Information is not made public regarding the academic proficiency of students who vanish from the cohort. What I do know is that dramatic rises in the percent proficient seem to parallel the rates of attrition in the testing cohorts . I have charted these in the latter part of this post.

Left or Left Back?

Clearly, testing cohorts are shrinking as proficiency rates rise. A question remains, however, as to whether students are leaving the school altogether, or being retained in earlier grades.

While either is possible, most of the schools under review have been in existence long enough to track not only the size of the cohort that the student is leaving, but also the size of the cohort he or she would enter if left back. If students are being left back, then their entrance into the cohort of the lower grade should be reflected in the size of that cohort. That cohort might grow, for example. What happens instead, however, is that those cohorts too are generally shrinking as students move up in grades.

Since the cohorts into which the vanishing students would be assigned are themselves shrinking, retention seems unlikely to be the major factor in cohort attrition.

However, if attrition is the result of student retention in the grade , then the sharply rising passing rates in these schools seem best explained by the elimination of struggling students from the cohorts.

A Closer Look at Four High Attrition Schools

Let’s take a closer look at attrition in the four schools that lose more than 25% of their students from the testing cohort between their first year in the school (grade 5, 2006) and their last year (grade 8, 2009).

The chart below lists the schools, their total attrition, and the proficiency percentage points gained across years:

ELA Cohort Attrition Grade 5 to 8 (2006-2009)


School ELA Cohort

Attrition

(number change)

ELA Cohort

Attrition

(% change)

Proficiency Percentage

Points Gained

Williamsburg Collegiate 72 to 44 -39% +35
Harlem Village Academy 57 to 39 -32% +43
Leadership Village 59 to 41 -31% +24
KIPP STAR 82 to 61 -26% +27

The graphs below aggregate the data in the chart above.

ELA cohort attrition grades 5-8

Next let’s compare the charters to neighborhood and similar schools, and then look at the possible relationship between proficiency and cohort attrition.

Comparative Attrition

Williamsburg Academy has 39% attrition: Meanwhile the neighborhood schools as a whole (CSD 14), and the similar school within the neighborhood (K582), have cohorts that do not decline. Overall they grow.

Williamsburg Collegiate

Harlem Village Academy has 32% attrition. It is in District 5 where the neighborhood schools as a whole and the similar school within the neighborhood (M317)[4] have cohorts that do not decline. Overall, they grow.

Harlem Village Academy

KIPP STAR has 26% attrition. Like Harlem Village, it is in District 5, where neighborhood schools as a whole as well as the similar school within the neighborhood (again, M317) have cohorts that do not decline. Overall, they grow.

KIPP STAR

Leadership Village has 31% attrition. Meanwhile the cohorts of the neighborhood schools (CSD4) do not decline.[5]

Leadership Village

Rising Proficiency

As students disappear, the high-attrition schools record ever higher percentages of proficient students. This same pattern plays out in all 13 schools. Schools with more than 20% attrition see their percent of proficient students rise to over 90%. We do not know if struggling students are in fact leaving the cohort. We do know that cohorts are shrinking, and rates of proficiency rise.

Here are graphs of our four schools with testing cohort attrition of more than 25%. Both attrition and percent proficient are able to be graphed together because the values in each scale are similar.

Rising proficiency

Notes

I reviewed the 15 charter schools the New York City Department of Education (DOE) identifies as middle schools on its Progress Reports. I excluded two of these schools from our analysis because they were new and therefore have only one year of data (I needed at least two years of data to follow a cohort). The two schools I eliminated are Bedford Stuyvesant Collegiate and St. Hope Leadership Academy.

I used the DOE’s data set for ELA test results for the 2006 through 2009 school years to determine cohort size and proficiency rates. I also examined the Math data which showed cohort attrition that was highly consistent with the ELA attrition.

Some schools have as many eight cohorts with more than one year of data between 2006 and 2009. In this report I examined the cohort of students who were in the school’s highest grade level (usually 8th grade) in 2009. Rates of attrition from the schools’ other testing cohorts is highly consistent with the attrition found in the cohort I examined.

  • Appendix One shows ELA attrition and proficiency in all thirteen schools for the cohort that I reviewed.
  • Appendix Two repeats the information in Appendix One and adds the next cohort.
  • Appendix Three shows the citywide aggregates for both cohort size and proficiency rates for the fifth grade (2006) to 8th grade (2009) ELA testing cohort.

Finally, I did not include the UFT Charter School in my review because it is a K-8 and I have focused only on schools designated as middle schools by the DOE. For the record, however, the UFT middle school cohort has significantly less attrition than the schools that are the subject of this study. One UFT middle school testing cohort has 15% attrition. The other actually grows.

All data can be found on the website of the New York City Department of Education at schools.nyc.gov.


[1] I considered all charter schools that have been designated as middle schools on the New York City Progress Reports.

[2] This is the 2006-2009 cohort charter middle schools typically begin in 5th grade, whereas regular public schools begin in 6th.

[3] Opportunity Charter, with a rate of 15% proficiency in 2009 has been eliminated from this average because its population of about 50% special education students makes it unlike the other schools. Its inclusion would lower the proficiency rate.

[4] A second “similar school” is M367, a new school which does not have a three-year cohort. Its first group of students declined from 81 to 77 students (a 5% decline) between 7th and 8th grade.

[5] The DOE has not identified any similar schools in Community School District 4.

Print

12 Comments:

  • 1 Christine Rowland
    · May 27, 2010 at 8:00 am

    A fascinating study Jackie – thanks. Is there any way to tell more about the composition of the students who left the cohort? Are we able to establish whether percentages of specific groups (such as special education or ELLs) went down?

  • 2 Gideon
    · May 27, 2010 at 9:35 am

    This is really important data, and should be scrutinized carefully. These are not cohort data but individual testing grades being compared year to year; we don’t know if it’s the same group of children from one year to the next. Students may have left and been replaced by other students enrolling in later grades. Also, charter schools may have different retention practices than the district schools: if they are holding back students in earlier grades who don’t meet grade level standards, one would expect a bulge in earlier grades and fewer students in the upper grades. Thus, it’s not necessarily attrition but instead could be retention that’s causing this pattern, which would also explain the higher scores in later grades. Finally, charter schools are choice schools, and parents get to decide whether to keep their children enrolled whereas most parents with children in district schools have few options, thus little opportunity for attrition in zoned schools. If children are leaving charter schools, it would be useful to know why their parents are choosing to take them out.

  • 3 Jackie Bennett
    · May 27, 2010 at 11:44 am

    Gideon thanks for the comments. In response.

    Are new students coming in? Are they the same students? Actually, we do know that in the charters no new students are coming in. The state web site tells us how many students in the cohort were there the year before. In the case of these schools, 100% of the students in the cohort that were there in 09 were there in 08 in these cohorts. To be perfectly precise, one of the schools seems to lose one student. By contrast, the regular publics take in new kids all the time, even when they are coming back mid-year from charters.

    Are vanishing students going to earlier cohorts, being left back? As I explained, that is not likely, since the lower cohorts shrink instead of bulge. As you pointed out, they ought to bulge. But if charters are holding students back to this degree, that would certainly say something about the percentages of passing students that remain in the grade .

    Regarding choices parents make: yes, absolutely, perhaps parents are choosing to pull their kids out of charters. One possibility is certainly that parents are voting with their feet.

    Finally, to Christine – Unfortunately, I cannot tell the demographic breakdown for the charter test scores, even though these breakdowns are easily available for regular public schools. As with many aspects of charter schools, information for charters is hard to come by. If you want to get discouraged click on the Special Education Delivery Reports for any charter, for example. Even if the link is there (sometimes it is), the page that comes up is blank.

  • 4 Dee Alpert
    · May 27, 2010 at 12:41 pm

    Can you get the NYCDOE discharge codes and numbers of kids from each charter “discharged” in each code – both for the charters and comparable NYCDOE schools? These are pretty detailed and might give more information as to where kids went after leaving their schools. Typically, for high school cohorts, the NYCDOE claims that the vast majority of kids transfer to school systems outside NYC. If this was true, at the published rates, NYC would be a ghost town. But for kids transfering between NYCDOE schools and between charters and NYCDOE schools, the data should prove revealing.

    Of course, NYSED doesn’t require that districts (or charters) have anyone audit and verify this data, but what you’ve got is the only available data and thus might be worth analyzing.

  • 5 Gideon
    · May 27, 2010 at 1:28 pm

    I’m not sure I’m understanding the data cohorts the way you are. Take Williamsburg Collegiate: I read your table to show 72 5th graders in 2006 and 44 8th graders in 2009, which you interpret as 39% attrition. Two main questions: 1) how do you know that all 44 students in 2009 were in the school’s 5th grade in 2006? For instance, couldn’t some of them have enrolled in the 6th grade in 2007. The important issue is the change in performance of students who were in BOTH the 2006 5th grade and the 2009 8th grade. That’s appropriate cohort analysis. 2) Why isn’t it possible that the decline in numbers was from retention? If 28 of the 5th graders in 2006 were held back, then only 44 would move on to subsequent grades and show up in 8th grade in 2009. Of course, its more likely that some were retained in 5th grade and others in 6th grade, etc. How many years of data exist for the upper grades? If retention is the issue, you should eventually see larger and larger upper grades as those students ultimately move up. Of course, retained students may be more likely to leave if they know that can go back to district schools with lower standards, which again could explain both attrition and rising scores at the charter school. Very complicated and hard to figure out without more data. I’m glad you’re pushing this issue. It would be very helpful if you would publish what ever spreadsheets or databases you’ve compiled to do this analysis.

  • 6 Bob Calder
    · May 27, 2010 at 7:33 pm

    It should be relatively easy to find the escapees/rejects at their respective public schools and interview them in order to reconstruct what happened. What were their scores? Why did they move? Do they know anyone at the former school who is willing to talk? I have found that parents sometimes move because of perceptions not facts, and since conditions are similar, see no reason to move back.

    I had a student move back to our school after leaving this year. He didn’t think the program was as good as ours. Who would have guessed? lol

  • 7 Patti Altman
    · May 28, 2010 at 6:11 am

    What evidence does the provide? These schools are located in areas with a high mobility population. The difference between the charter school and the neighborhood school is that neighborhood schools fill open spots in the cohort with new students where as charters generally do not accept students in the cohort past 5th or 6th grade.

    If the neighborhood schools chose not to fill spots left vacant by children moving out of the neighborhood they would see similar rates of attrition.

  • 8 Jackie Bennett
    · May 28, 2010 at 11:36 am

    Patti – If I can find rates of mobility in NYC schools I will post them, but I can tell you this: they are not near what we see in these charters.

    But the much more important point is that our public schools cannot “choose” not to accept students into their cohorts. We accept students even when the classes and the hallways are busting at the seams. We accept them mid-year, from the homeless shelters, fresh from immigration. Not our choice, and to my mind no school should be able to make that choice, though of course the charters can.

    That makes for a very different condition and less time for us to make an impact on their education, even though our schools are judged on the results.

  • 9 John S
    · May 28, 2010 at 10:21 pm

    Jackie, I think you might be able to slice these numbers in a way that’s even more revealing. Consider the first set of graphs for the aggregate 5th grade cohort. In 5th grade, there are 270 students and the proficiency rate is 60%. That’s 162 students. In 8th grade, there are 180 students and the proficiency rate is 90%. That’s 162 students.
    Unless I’m missing something, the impressive-seeming percentage gains appear to be entirely the result of attrition.

  • 10 T.
    · Jun 1, 2010 at 2:19 pm

    Yes, these are high mobility areas. BUT if I, as a parent, had to sweat through a lottery and put in the effort to keep up with the requirements of a charter school, I’d be pretty unlikely to move to a place where my kid could no longer attend that school. Unless, of course, I was unhappy with that school and wanted out.

  • 11 Augmenting the UFT’s “Vanishing Students” Report | GothamSchools
    · Jun 9, 2010 at 3:32 pm

    […] was very interested to read the UFT’s latest report on charter school attrition in middle schools, as I’ve had trouble finding reliable […]

  • 12 HechingerEd Blog | Are charter schools succeeding because they hold students back?
    · Jun 11, 2010 at 12:12 pm

    […] study corrects a study by the New York City teachers union, the United Federation of Teachers, which found that charter schools have high rates of attrition […]