Earlier this week, the National Center for the Study of Privatization in Education [NCSPE] at Teachers’ College published an important paper by University of Illinois professors Christopher and Sarah Theule Lubienski, Charter, Private, Public Schools and Academic Achievement: New Evidence from NAEP Mathematics Data.
The NAEP exams are without question the best comparative measures of student achievement we have in American education. Unlike the various state exams produced for NCLB purposes, the NAEP assessments are not high stakes tests, and students do not undergo months of test preparation before they take them. Consequently, psychometricians find that the results of the NAEP exams are more reliable than those of the high stakes tests. Moreover, the various state exams are quite uneven in quality, while the NAEP tests are consistently good. The NAEP tests are simply more valid exams. Finally, NAEP assessments are based on a single, rigorous set of standards, while many state exams are notorious in the low standards they employ.
But even data from the best of tests can only tell you so much about student achievement and the quality of education. Students come to schools – and to tests – from a variety of backgrounds, and when it comes to learning, those backgrounds give some students extraordinary advantages and other students immense disadvantages. The greatest challenge in assessment today is how to separate the effects of the education provided in the school from the effects of background. When all the posturing was done, this was the one real issue in the Charter School Dust-Up over the relative performance of district public school students and charter school students on NAEP – to what extent could the poorer performance of charter school students be attributed to the background of the students, as opposed to the education provided by the schools.
This is why the analysis of the Lubienskis is so important. Building upon their earlier work with previous data sets from NAEP Math exams [see, for example, this essay in the May 2005 Kappan], the Lubienskis used a sophisticated statistical analysis that allowed them to control for what they call demographic characteristics [socio-economic class, race, gender, English Language Learners, disability] and school location. With this control, a rather different picture appears.
In fourth grade Mathematics, most school sectors bested district public schools in the raw scores: Catholic school students scored 9.5 points better, Lutheran school students scored 10.7 points better, conservative Christian school students scored 4.2 points better, and non-sectarian private school students scored 11 points better. Only charter school students scored more poorly, by 6.1 points. But when controlled for demography and school location, the picture changed dramatically: the district public school students scored 7.2 points better than the Catholic school students, 4.2 points better than the Lutheran school students, 11.9 points better than the conservative Christian school students and 5.6 points better than the non-sectarian private school students. While district public school students continued to score better than charter school students, the gap shrunk to 4.4 points with the controls in place.
A similar pattern is found in eighth grade Mathematics. In the raw scores, Catholic school students scored 14.3 points better, Lutheran school students scored 21.2 points better, conservative Christian school students scored 5.4 points better, non-sectarian private school students scored 14.3 points better and charter school students scored .9 points better. When controlled for demography and school location, the district public school students scored 3 points better than the Catholic school students, 10.6 points better than the conservative Christian school students and 2.3 points better than the non-sectarian private school students. The gap with Lutheran school students fell 20 points, to 1 point, and the gap with charter school students increased to a 2.4 charter school advantage. The Lubienskis report their most significant findings as the following:
Public schools significantly out-scored Catholic schools (by over 7 points in 4th grade, and almost 4 points in 8th grade).
Of all private school types studied, Lutheran schools performed the best. 4th grade scores in Lutheran schools were roughly 4 points lower than in comparable public schools, but were (a statistically insignificant) 1 point higher at the 8th grade.
The fastest growing segment of the private school sector, conservative Christian schools, were also the lowest performing, trailing public schools by more than 10 points at grades 4 and 8.
Charter schools scored a significant 4.4 points lower than non-charter public schools in 4th grade, but scored (a statistically insignificant) 2.4 points higher in 8th grade.
Interestingly, John Stossel got in touch with the Lubienskis when he was preparing his Stupid in America report, but their research was just one more inconvenient truth.
19 Comments:
1 nycparent
· Jan 29, 2006 at 7:44 am
What i find bizarre about this study is the little words “when you account for demographic differences” The public schools do worse on the tests than both groups, but then add some statistical analysis that says…well, in hypothetical land, where you could be educating rich white kids, you guys would do ok. That makes the AFT proud? What use is this in reality. The fact is the public schools educate a different type of kid and they aren’t doing it well.
2 NYC Educator
· Jan 29, 2006 at 9:24 am
The fact is they do a damn sight better than private school teachers could ever do. Set them in a local NYC high school and see how long they last.
Comparing a group of kids with 100% proactive parents to simply everyone is preposterous. With a mayor who adamently opposes high standards for teachers and low class size, with a governor and mayor who’d rather fight about money than fund schools to the levels of their neighbors, it’s a minor miracle the schools are even standing.
You elected this mayor, no one doubts he bought the election fair and square, and even the UFT, Lord knows why, supported this governor.
You get what you pay for. You want a tax rebate on the $3,000 you pay on your million-dollar home? Fine. I was priced out of the district I work in 13 years ago, I pay double the taxes on a home worth less than half, but when I visit my daughter’s school I know exactly what I’m paying for.
Travel 10 miles east to see what education looks like when it’s supported by the community.
Or you can just keep blaming the teachers. Good luck with that.
3 Leo Casey
· Jan 29, 2006 at 11:00 am
There is a willful refusal on the part of “NYCParent” to come to grips with the argument here. There is no claim here — nor do I think one can find such a claim anywhere in the main posts on Edwize — that public schools can be satisfied with what we do. Indeed, our message on Edwize is one of how public schools must change, given the demands of the new knowledge economy. We simply can not be content with providing the same quality of education we did five decades ago, when a high school dropout could still obtain a decent paying industrial sector job, especially if those jobs were unionized. Good-paying, unionized industrial jobs are now on the endangered species list. We need to produce the sort of quality education which gives our youth real life opportunities in the 21st century global economy.
The issue here is what kind of change will improve public education — and the life opportunities — of young people who are born into poverty. What the Lubienskis’ analysis does so powerfully and convincingly is demonstrate that the simple-minded ‘magic bullet’ ideologies that say all we need to do is privatize public schools, to adopt vouchers, to employ market strategies and so on have no basis in the real world of American education. When they are teaching the same students from the same backgrounds as district public schools and charter schools, private schools do more poorly. Charter schools do better than private schools, but they are still not performing on the same level as district public schools. So these magic bullet laissez-faire market strategies would only diminish the education and the opportunities of our youth. They are the work of ideological ‘true believers,’ not educators.
The hope for young people living in poverty is change within and improvement of public schools.
4 NYC Educator
· Jan 29, 2006 at 11:14 am
“They are the work of ideological ‘true believers,’ not educators.”
Sometimes, yes. But wealthy idealogues promoting vouchers have no real interest beyond eliminating public schools and the consequent inconveniences they cause Steve Forbes’ tax bills.
Or are we to believe those who cry screw the Wal-Mart employees, screw the teachers, screw the TWU, screw working people everywhere, screw their pensions and benefits, but we adore working-class kids and passionately want a good education for them?
5 Chaz
· Jan 29, 2006 at 11:22 am
nycparent;
In plain english the private and charter schools don’t work unless these schools employ all of the following procedures:
* Low class sizes
* expelling discipline problems
* parent involvement
These procedures cannot be mandated in the public schools. Therefore, you do have a gap in test scores in some non-public schools that employ the procedures. However, the reason that there is not a significant gap in the test scores is probably attributed to the experienced teachers that work in the public schools who use time-tested techniques that help deal with the large class sizes, discipline problems, and lack of parent involvement.
I’m sure your child is a good student because of your involvement in his/her education.
6 nycparent
· Jan 29, 2006 at 4:11 pm
Leo Casey, I surely didn’t see the union representatives saying “we must do better” I read them patting themselves on the back, as usual. Also, as we all know in the public schools, the most crowded public schools in this city are the best performing. I don’t buy the low class size argument, as paramount. Don’t get me wrong, I’d love 5 kids in a class but I don’t see it as panacea. I think discipline is definitely an issue. Parental involvement, from my point of view, is also a farce. My friends in private schools have no clue what’s going on in the classroom. They don’t have to show up to all these meetings, and they certainly don’t need to tutor like we do. This is another common parental joke. Private school parents are allowed to send them off and assume they are being educated. Now that IS a luxury.
7 Chaz
· Jan 29, 2006 at 4:55 pm
nycparent;
You are wrong! Every independent study on class size has shown that smaller class sizes results in higher test scores. Further, many other studies link student success with parental involvement. Maybe your friends in private school figure paying $20,000 per year in a private school is the reason their children do well. If the students don’t do well or act up they will be expelled from the school. In other words, there are consequences for the students action.
By the way just think how much better the good schools would be if we could eliminate overcrowding!
8 NYC Educator
· Jan 29, 2006 at 6:09 pm
“Private school parents are allowed to send them off and assume they are being educated. Now that IS a luxury.”
I’d liken it more to idiocy, actually. I wouldn’t trust my kid to the good graces of any administration, public or private.
Furthermore, to suggest that parents who make the decision to send their kids to private school, then pay thousands to do so are somehow uninvolved is equally nonsensical.
Some crowded schools perform well, until the next disastrous fire. That’s what it will take to end overcrowded schools under this mayor.
Also, there are plenty of crowded schools that do not perform well. You are sorely misinformed. There are also plenty of “small academies” crammed into overcrowded schools, which makes them overcrowded as well.
Parental involvement is the single best predictor of a child’s scholastic success, or lack thereof. If you truly see it as “a farce,” I can only pity your child.
The second best chance, for kids with parents who think involvement is “a farce,” is good teachers, something NYC hasn’t demanded for 30 years, despite the Chancellor’s noise machine. We continue to maintain the lowest standard for hiring and retaining teachers in the state.
After that, I’d say, small classes is the icing on the cake. We have the highest class size in the state. And once again, if you think parental involvement is “a farce,” and that good teachers and small classes are unnecessary, be proud to live in NYC, and keep supporting Mayor Mike..
Because lucky you, you’re getting precisely what you asked for.
On the off chance you want to know what’s really going on, you might try going here:
http://www.classsizematters.org/
9 Leo Casey
· Jan 29, 2006 at 6:59 pm
NYCParent:
1. The evidence regarding the efficacy of smaller class size is quite strong and powerful. Moreover, one of the important things that have been learned from large scale studies of the effects of small classes in Tennessee [the STAR program] and Wisconsin [the SAGE program] is that smaller classes have a particularly beneficial effect for children living in poverty and children of color. A lot of the evidence is easily available on-line.
See the summary of the educational literature on the subject in the American Education Research Association’s Fall 2003 Research Points [Volume 1, No. 2], “Class Size: Counting Students Can Count.”
See also the accounts of Jeremy D. Finn, Et. Al., “The Enduring Effects of Small Classes” in Teachers College Record, Volume 103, Number 2, [April 2001]. pp. 145-183.
See also Lawrence Mishel and Richard Rothstein, editors. The Class Size Debate. Washington, DC: Economic Policy Institute, 2002.
See A. B. Krueger and Diane Whitmore, “Would Smaller Classes Help Close The Achievement Gap?” Princeton University Department of Economics Paper. [March 2001]
See Phil Smith, Alex Molnar and John Zahorik, Class Size Reduction In Wisconsin: A Fresh Look At The Data. Arizona State University: Educational Policy Research Unit, September 2003. (An abridged version of this report was published in Educational Leadership [September 2003].)
That should give you enough bed time reading for a week. {-;
2. I would be interesting if you could point to a concrete instance of union representative on Edwize “patting themselves on the back” on how good public education is doing. Does it pain so much to admit that public schools are performing better than the private alternatives that the only answer is to suggest that an accurate accounting of that matter constitutes a denial of the need for public schools to do better?
It is one thing to hold a view in the absence of being aware of contrary information; it is quie another thing to continue to hold it after the contrary information has been made available.
10 nycparent
· Jan 29, 2006 at 8:13 pm
I am quite aware of all the class size studies. I presented a comprehensive view of the issue to the SLT at my school. The conclusions after viewing these studies and others that paint a different picture is that small class sizes do help in some circumstances if you keep all variables constant. Problem is that in order to move the needle in these instances, the number of students in a class, (especially poor and urban) would need to be so small, that it is almost impossible to imagine this being sustainable. More importantly, the class size effect is null and void if you account for one other variable: quality teaching. Instead of classsizematters.com, i’d like to see goodteachingmatters.com. Yes, of course it’s easier to be a good teacher with 15 kids but i’d still take a good teacher with 30 kids over a lame one with 15 kids. Any parent would. Regarding parental involvement, my point is that for involved parents, the amount asked of us by public schools is staggering, compared to our brethren in private. We tutor, fundraise, volunteer, buy supplies, teach supplemental materials, make up for the holes in the curriculum and the lack of teaching quality and the outcome for us is that the kids are… kind of learning.. something… sometimes. For my ever-patient public school friends, the student achievement does not match our effort. I wonder what could account for this gap? As for uninvolved parents, i suggest the whining and excuses stop and some radical new approaches are tried. What you are doing isn’t working. You know the definition of insanity, right? Doing the same thing over and over and expecting a different outcome. Finally, frankly no, i don’t really hear a lot of hand-wringing on this blog about performance and teacher excellence. It mostly is about blaming others.
11 john
· Jan 29, 2006 at 9:24 pm
I don’t know how we got on to class size. I guess small class size is part of the reason some parents choose private schools. But this study is more useful for policymakers than for parents because it’s about the average charter school, the average public school and the average private school.
Parents choosing a specific private school? Of course that might be the best thing for their kid. Policymakers pushing the rapid expansion of vouchers or charter schools? Not good for the kids.
12 NYC Educator
· Jan 29, 2006 at 9:25 pm
“i’d still take a good teacher with 30 kids over a lame one with 15 kids.”
Me too, and I said that before. Good teachers come first.
I’ve also said that when you have the lowest standard for teachers in the state, you are hardly trying to attract the best.
Mayor Bloomberg took the LAST test, a basic competency test, passed it, and declared any high school graduate ought to be able to pass it.
He then sent Chancellor Klein to Albany to beg for the right to retain and continue hiring those who failed it.
In my suburban district, hundreds apply for each opening. In NYC, despite lower standards, 800 numbers, websites, job fairs, and unbelievable worldwide recruiting efforts, they’re lucky indeed to find one.
Doing the same thing over and over and expecting a different outcome? No, I don’t endorse that at all, nor did I say so.
You do not read very carefully.
There are certainly ways to attract good teachers. Mayor Bloomberg chooses not to use them.
Now, a moment ago parental involvement was a “farce,” but now you’re running bake sales, tutoring, volunteering and doing God knows what else–little of which has much to do with parental involvement anyway, which occurs mostly in the home.
That speaks to your credibility.
You voted for Mayor Bloomberg. You wanted status quo, and now you’ve got it. What on earth are you complaining about?
So there are bad teachers. The UFT does not do the hiring.
You do.
13 Leo Casey
· Jan 29, 2006 at 9:28 pm
Everyone is entitled to their own opinion, NYCParent, but not their own facts. Your account of the class size issue may justify your own predilection to dismiss the issue, but it does not accurately reflect the state of the research. The very best research on the issue is based on statewide programs, the STAR program in Tennessee and the SAGE program in Wisconsin. One of the reasons why that research is so powerful is the sheer size and scope of the programs. The only sustainability issue here is whether or not elected officials sustain the will to support small class sizes.
No one at Edwize denies the importance of teaching quality in education. I have made that point here on a number of occasions, as have others. I can only assume that because we made that point in terms of the need for quality professional development, for self-policing of the profession, etc., and you aren’t concerned with those matters, you just passed over it.
As for ‘blaming others,’ ‘whining’ and ‘excuses,’ I suggest you go back and take a look at the original post and then at your comments.
14 Chaz
· Jan 29, 2006 at 9:39 pm
nycparent;
I agree quality teaching is the most important aspect in the classroom. However, I also believe the NYC school system has many quality teachers and these quality teachers (many on this blog) will tell you that you are wrong about class sizes, and parental involvement.
For you to ignore all the studies that have been done on these two issues and to rant about the few teachers in your school that may not meet your standards is ridiculess. It’s the many quality teachers in the NYC school system that gives the children a chance.
Blame the Bloomberg/Klein administration who hire the teachers in your school and the school administrators who gave them tenure in the first place. Further, blame the Bloomberg/Klein administration on shortchanging the schools in money and supplies and allowing the overcrowding of good high schools by their fixation on charter/small schools that do not work!
Finally, you need to keep an open mind on what works in a school not listen to the right-wing zelots who don’t care about the public schools or the left- wing teacher college crowd who want to micromanage every minute of classroom instruction.
If you want to improve your child’s education? Then support the “Let Teachers Teach” program and ensure that the schools have adequate funding.
15 nycparent
· Jan 30, 2006 at 6:39 am
NYC Educator: Point of clarification, I said parental involvement, as it related to an excuse for poor school performance was a farce, not that parental involvement was a farce. YOu need to understand nuances and inferences in your reading. You must not have been TC taught.
16 NYC Educator
· Jan 30, 2006 at 7:46 am
“YOu need to understand nuances and inferences in your reading.”
Actually, for anyone to do that, you’d need to express them.
You said, completely and exactly exactly “Parental involvement, from my point of view, is also a farce.”
You are incapable of a substantive response, so you nitpick over nonsense.
17 northbrooklyn
· Jan 31, 2006 at 8:34 pm
How do I email this article to others?
18 divina
· Feb 1, 2006 at 10:55 am
I’d just like to interject an idea while ya’ll are defending your profession.
When comparing funding between our schools, vs. international schools, keep in mind that part of the money we spend is spent on:
* School meals
* Busing
* Health care (school nurses etc.)
Now if we had better safety nets, a lot of this funding would be unnecessary. Even without the safety nets, one must subtract the costs for these extra, non-educational expenditures to get a more accurate picture of what we actually spend on education.
19 CityTeacher
· Feb 1, 2006 at 7:29 pm
We tutor, fundraise, volunteer, buy supplies, teach supplemental materials, make up for the holes in the curriculum.
poor nycparent, imagine the horror of having to take care of your own children and look after their educational needs. Lack of parental involvement leading to poor performance is certainly not a farce or an excuse, but one of the most accurate cause/effect predictors of student acheivement. If you were in our field and read the studies we read you would know this, since this is our job and area of expertise and we know what it takes to produce a good student. It always amazes me how many “experts” there are who can do our job so much better than we can who have never been in a classroom. Without parental involvement (checking homework, checking notebook, knowing when a test is given and what the kid gets on the test should be at least the minimum) the student is not going to make it. If you resent having to be on top of your kids schoolwork maybe you shouldn’t have had them.