Archive for the ‘Testing’ Category
Here is a headline that was missing in the NYC papers this past week:
NYC Teachers: The Best Teachers in the State.
And here is the missing lead to the article.
An analysis of New York State’s growth scores reveals that NYC teachers are twice as likely to be considered Highly Effective compared to teachers in the rest of the state, and about half as likely to be Ineffective. That analysis is based on the results of the state’s teacher growth model and this year’s new math and reading tests.
And here is the missing sidebar:
2013 Growth Score Results
Percent of Teachers in City
Percent of Teachers in
Rest of State
True, that story would be based on test scores in ELA and math, grades 4-8. True again, the state used its statistical growth model formulas to arrive at those results. And, true a third time, test scores and growth models can never be the only measure of teacher effectiveness.
But still. When have those limitations ever stopped the press from publishing test-score stories about teachers in the past?
For example, two years ago, it was front-page news when some researchers tied the “quality” of 4th grade teachers to the marginally increased incomes of their students two decades later. We are talking here about a single study that made a cause and effect link between two events happening 20 years apart, and a salary increase of a few hundred bucks a year. Is that front page news? Yet there is was, and it got the intended traction, too — trotted out at dozens of forums nationwide as a justification for firing teachers based on their students’ test scores.
That’s not the only example, of course. When schools — and implicitly their teachers — are labeled F’s and D’s based on test scores, the press is happy to carry those stories.
And let’s not even discuss all the eagerness around the value-added TDRs.
But when it comes to news that essentially says “Let’s stop the war on NYC teachers” ? Nothing made the printed press. In fact, if it hadn’t been for the folks at Gotham Schools, we may never have known.
But to return to the findings on our teachers. Here is what else the New York Post, or Times, or some other paper could have said:
Only one in 20 teachers in the rest of the state was found to be “highly effective” — but in New York, that number was one in 10 (11%). And while the rest of the state had more Ineffective teachers than Highly Effective ones, in the city there were three times more Top Teachers than struggling ones according to their scores.
NYC math and reading teachers earned those results in spite of the intense challenges that their students often face. More city kids arrive at school learning disabled, poor, and new to the English language.
These findings are based on a very large data pool of data (about 40,000 teachers and well over one million kids). And this is actually the second year that city teachers outperformed the state. Last year, the differences seemed smaller, but researchers attribute the widening gap to improvements in the statistical models, which better capture the results.
Really, and truly, I do know how scary it can be to validate our existence with any of that.
But in a world that is perfectly willing to debase us through scores — why can’t the world out there extol our virtues, too?
So, congratulations to our teachers, and our schools. We know that tests can’t begin to capture what you do, and that growth models can’t capture all the challenges we face. But they do say something — and it bears repeating in a public space.
A few months ago we did an analysis of the college-readiness numbers of the system’s high-school graduates. That analysis showed that the system’s college-readiness rate – 22.2% — was largely attributable to a small number of high schools. The 35 schools that made up the top 10 percent of high schools contributed nearly half of the graduates ready for college, while hundreds of other high schools had college-ready rates in the single digits.
That same pattern – a “tale of two school systems” – is echoed in the results of the recent state tests on grades 3-8 based on the Common Core standards. In these most recent tests, as in the college-ready statistics, a minority of high-achieving schools helped camouflage much lower results in the majority of schools.
In these new tests based on the much more demanding Common Core standards, only about 26 percent of grade 3-8 students citywide were judged proficient in reading and 30 percent proficient in math, far below the achievement levels on previous state tests. But even this unimpressive level was not reached by the great majority of elementary and middle schools. In these schools student achievement was lower – sometimes much lower — than the citywide average.
One quarter of schools produce half to two-thirds of proficient students
Another way to think about this: About 175 elementary schools – out of a total of 600 – educate at least half of the city’s proficient math and ELA students. Of the city’s roughly 350 middle schools, only 90 account for more than two-thirds of the city’s proficient math and English students.
So a student at one of these schools has – mathematically at least – an excellent chance of passing these tests. But the opposite is also true. Students at the remaining hundreds of schools face much larger challenges, particularly at the roughly 275 schools where only 10 percent of children – or fewer — scored proficient in either reading or math.
Same old same old
The Bloomberg administration has consistently reported the numbers in a way that uses an “average” to de-emphasize the problems at hundreds of low-achieving schools. While the new Common Core tests have changed the definition of proficiency, their results are consistent with the pattern of achievement – and underachievement — for so many New York City schools.
When the city’s 600 elementary schools are ranked into top, second, third and bottom quartiles by their 2013 scores on the Common Core tests, more than 70 percent land in the same quartile they did based on the state tests from 2012.
For the 352 middle schools with two years of math results, 68 percent maintained their rank. Even those that changed ranking generally moved just one quartile: half of those that changed went up one quartile, and 43 percent went down one. Only five schools moved two quartiles up or down.
With this year’s introduction of Common Core-aligned tests, flawed as they were, the city schools enter a new era. The transition will be a game changer that will bring angry reactions by teachers and students, and wider class and racial performance gaps. Student achievement measurement may become discredited for awhile, as an exasperated public throws up its hands in confusion.
Those could be the best things to happen to standardized testing in 10 years.
Achievement plummeted on this first try at new tests. City students scored 20 points lower in ELA and 30 points lower in math, with less than 30 percent of Grade 3-8 students meeting proficiency standards. But the tests set an extremely high bar — probably too high — in what amounted to a premature effort to test students against the new Common Core. Curriculum didn’t start to be available until late in the year and the Dept. of Education didn’t have the leadership required to manage such a dramatic transition.
But there is no going back. The New Common Core tests, which will continue phasing in over the next few years, may get better, especially if current state test-maker Pearson PLC moves out the way. But they will remain harder: they will ask students to do more explaining, analyzing and creating.
And here’s the thing: these are the very skills educators want to teach and have had to forego in favor of test prep. Right now, teachers are out of practice, and so are their students. But these are the skills they want to teach. So they will demand more autonomy, an end to the culture of test prep, more time and resources. As long as the state and city don’t slap ridiculous consequences onto the new scores, students and teachers alike will become less bored and hopefully more engaged.
Harder tests are going to result in widening gaps between better- and less-prepared students. Typically that means racial and wealth gaps, as well as gaps between English proficient and ELL students and between general and special education kids.
These were the gaps that No Child Left Behind set out to eliminate back in 2002. To the extent this succeeded, and it didn’t much, the cost was relentless test prep and/or dumbing down of tests. Now, as the gaps widen on the Common Core tests, parents will be outraged and politicians will distance themselves from the schools. So they should. Bringing poorly prepared students up to standards is the work of brilliant and passionate teaching, which has been forced underground in the NCLB era. Its reemergence can come only if good educators are free to work. They cannot be commandeered by mayors running numbers. A next generation teacher force can only be brought into being by experienced educators who are not ruthlessly tracked by narrow performance monitors.
Accountability and Legacy
The education mayor, the education president. These monikers turned out to be albatrosses around the necks of Michael Bloomberg, George Bush and many others. Their legacy is a culture of measurement, not of learning. Testing has become laden with consequences that the tests themselves were never meant to support, including judgments about schools, teachers, and even “where we are heading as a society.”
One of the best things these new tests could do is force accountability to grow up. The city has overwhelmed us with data that, on close examination, is really the same data points parsed a hundred different ways. What’s more, the numbers appear to lie, or at least, they zing up and down without apparent reason.
Common Core tests could do two things about accountability. The first is to force us to adopt a more rounded assessment of students and schools. The second is to put standardized tests back their rightful, and less overblown, place.
So less than a third of students meet standards. Well, what else do we know? How do students perform on social studies projects, lab work, art and music, sports, leadership activities, group tasks, or community service? What 21st century skills do they have; what ones need to be developed? What are the best models for teaching those skills? What can students tell us about what they do and don’t understand and what helps them learn? And how do we measure those?
There needs to be some opening up — more quantitative data that uses non-numerical measures. We have agreed that more than half of teacher evaluations will be based on observations of classroom performance. Why can’t we assess our students that way?
It would be a relief if tests become more the province of educators. Politicians don’t find scales, cut scores, p values and item analysis inherently sexy. But good measurement requires expert interpretation. If the heat gets turned down under testing, and we all agree it’s complicated, then public attention may return to subjects, to projects, to school activities and to learning.
Of course, there’s another scenario, in which the new tests are simply misused as the old ones were, to pass judgment based on partial evidence, to bash and shame and to claim undeserved legacies. But after a decade of this, teachers and parents, not to mention students, are pretty fed up. Their voices lend hope for a turnaround.
This will be the fourth year that my students and I have suffered through the New York State high-stakes elementary school tests. Although the mayor and the chancellor tell us this year’s tests are all new, my stories from the classroom are similar to years past.
As a new teacher and New York City transplant, I was astonished to discover 3rd-, 4th- and 5th-grade students were held over based on their scores from a series of limited assessments. After that realization, I was much less surprised to see the effect of these tests in the classroom. Both schools I have worked at ended regular instruction in early February to opt for test prep units designed to milk a few extra points on the state exams. Students’ and teachers’ health began to slowly decline around the same time of year, and behavioral incidents began to rise.
In my own classroom, I have fought to ameliorate the stresses of testing season by reminding my students how hard they have worked and telling them that their only job on state testing days is to try their best. But my efforts have been less than successful. One year a 9-year-old 4th grader asked me if it was okay to put the classroom trash can near her desk in case she got sick to her stomach during her English language arts exam. The next year a mental block caused a little boy to flip his desk over in a moment of panic and frustration while trying to craft an extended-response essay. Just last week, Natashi, a girl in my 5th-grade class who has only been in the country for two years and is still transitioning to English, asked me whether I would be disappointed in her if she tried her best and still wasn’t able to pass. “What if I just need another year in 5th grade to keep practicing, Mr. Thompson?” she said to me with tears in her eyes.
With a broken heart and tears in my own eyes, I turned to Natashi and told her I would always be proud of her. “You have fought so hard this year! I will be proud of you no matter what score you get!” Natashi feigned a smile and asked to go to the bathroom to wash the tears off her cheeks.
My students, Natashi included, have been attending an extended-day program on Tuesdays and Wednesdays after school all year long. We have spent the last few months keeping students late on Mondays and Fridays for an hour and a half of extra instruction focused on test sophistication. For the past two months, we have asked students to come to school from 9 a.m. to noon on Saturdays for extra help to boost scores on their state tests.
Still, all the Common Core-aligned data I collect are telling me that my students are not showing mastery on the vast majority of Common Core standards. Many of the “grade level” reading passages and math problems I share with my students are far beyond their ability levels. The confusion these tasks generate leads to an overwhelming sense of failure among my students. And, of course, when my students feel like they are failing, I feel like a failure myself.
Should it surprise any of us that high-stakes tests, coupled with new standards, little-to-no teacher training, and no citywide curricula are a recipe for disaster? Should cheating scandals, state test boycotts, low teacher retention rates, and teary-eyed students come as a shock to the American educational system? Should I be surprised that my students score 30 percent lower than last year, as predicted by many educational experts? No!
The only surprising part about this whole process is the process itself. We have created a demoralizing atmosphere of fear, frustration and failure for teachers and students. I will always be proud of the hard work my students put into their education, and I sincerely believe they will succeed regardless of what their state test scores suggest. But if the mayor or the chancellor were ever to come up to me like Natashi did to ask whether I was proud of the reforms they had made to education, my answer to them would be quite different from my answer to her.
Mr. Thompson is the pseudonym of a fourth-year elementary school teacher in Brooklyn. A version of this post first appeared on the UFT blog edwize.org, where “New Teacher Diaries” is a regular feature. If you’re interested in writing a New Teacher Diary entry for edwize, send an email to email@example.com.
[Editor's note: This post was authored jointly by Maisie McAdoo and Rhonda Rosenberg.]
Mayor Bloomberg turned the announcement of the 2012 state test results into a promotional event for his “reforms” on Tuesday, despite the fact that an honest appraisal of the scores showed that city students as a group made only modest progress in both math and ELA. The mayor’s presentation ignored or downplayed results that didn’t fit in with his triumphal narrative, including the fact that the racial achievement gap widened last year in a number of categories.
State officials, by contrast, didn’t even hold a press conference, and said publicly only that the statewide results (which mirrored the city’s) showed “some positive momentum” but left too many students unprepared.
The mayor, however, orchestrated a big press function and handed out a shameless PowerPoint that reported highly selective numbers and featured a comparison of charters and new schools founded during his tenure with “traditional” city schools — i.e. the vast majority of schools in the city system.
But the numbers are there for all to see. “His” charters and new schools combined underperform the average school, in fact (see especially slide 6), and they gained only one to two points more than the “traditional” schools in percentages of students meeting standards in math and less than a percentage point in students meeting standards in English. That, according to the mayor, was conclusive evidence for the success of his reforms.
Please. If these test scores — and remember this was the testing round where 30 questions had to be disqualified, the same round that included the “pineapple” passage — if these scores are evidence of Bloomberg’s triumph as steward of the city school system, then pineapples can speak.
Here are some tables and charts on the 2012 tests that may sober up Hizzoner: More »
Click on the graphs for larger versions.
It’s standard fare to question the effects of high-stakes testing on education. Mention tests and people will tell you about the narrowing of the curriculum, the lowering of the standards, and the changed understanding of what it means to be educated, which used to have something to do with pleasure and imagination and now has mostly to do with isolated, testable skills. But in New York City, testing is not the only thing that is high stakes, and as it turns out, there are more ways to dumb down an education than to add another high-stakes test to it. Consider, for example, New York City’s high stakes credit accumulation scheme.
In this city, the number of credits awarded to students in high schools truly is high stakes. It counts as nearly one third of each high school’s Progress Report grade, and the Progress Report counts for just about everything, including the removal of principals and the closing of schools. Since the Progress Reports were introduced in 2006-2007, the percent of students earning 10 or more credits each year has leapt a (truly) incredible 16 percentage points citywide. For schools with the highest concentration of high need students (the schools most likely to be threatened with closure) the jump is 18 points. Most schools accrued those gains between the first and second years of the Reports. More »
In today’s New York Times is a letter from Diane Ravitch in which she responds to David Brooks’s recent column about testing and charters. She begins:
Mr. Brooks has misrepresented my views. While I have criticized charter schools, I am always careful to point out that they vary widely. The overwhelming majority of high-quality research studies on charters shows that some are excellent, some are abysmal and most are no better than regular public schools.
Some charters succeed because they have additional resources, supplied by their philanthropic sponsors; some get better results by adding extra instructional time. We can learn from these lessons to help regular public schools.
The Times adds this note below the letter:
We invite readers to respond to this letter, as part of our new Sunday Dialogue feature. We plan to publish a sampling of responses in the Sunday Review, and Diane Ravitch will be given an opportunity to reply. E-mail: firstname.lastname@example.org
As NY and other states gear up to spend their Race to the Top funds on developing more and more standardized tests and curriculums focused on passing them, here’s a powerful cautionary tale from Detroit about where that money may (vs. should) be going.
What did those of us staying in the same downtown hotel as Hollywood stars like Samuel L. Jackson think we could do to help fix the Detroit Public Schools?
I asked myself that question at the time, and I ask it especially now, when I’m amazed to read about the “draconian” measures being taken by DPS Emergency Financial Manager Robert Bobb. While he concedes his plan makes neither financial nor academic sense, Bobb is nonetheless attempting to solve DPS’s $327 million budget shortfall by closing nearly half of Detroit’s schools and increasing class sizes in the remaining ones to as high as sixty. It seems an insane idea to me, especially since I feel responsible for a big chunk of that deficit.
The company I worked for, you see, is owned by Houghton Mifflin Harcourt, and Houghton Mifflin Harcourt just completed a 15-month contract with the Detroit Public Schools worth $39,859,925.00. That’s right, almost forty million dollars, or more than 12 percent of DPS’s entire budget shortfall, for HMH’s “managed instruction” in reading and math. While I don’t know exactly what forty million dollars of “managed instruction” looks like (who does?), I know some of those millions were used to pay for the tests I helped slap together (mostly recycling passages and questions from our item bank that had been used many times before) and to sponsor my travels to Detroit.
Incoming kindergarten students have already been pushed into cutthroat competition against their peers. The coveted trophy is a spot in one of Chicago’s 500 allocated slots for classes for the “gifted.” There are more than 3,330 entries so far, and parents are fighting with all their resources and ingenuity to give their own kid an advantage in this Darwinian survival challenge.
They are coaching their kids themselves and those who can afford it have engaged professional tutors to plant test-acing strategies in their kids’ minds. They are racing to figure out how to crack the secrets of the test so that their kid beats the next guy’s kid to the fast track.
The training centers are commonly referred to as “boot camps.”
This story, reported recently in the Chicago Tribune, provides a perfect example of the pathological role that testing is playing these days. Not only does it substitute for curriculum, it also spoils and supplants the beautiful innocence of childhood. More »
It’s true, in a sense, that all that happened Wednesday was the state reported test scores using a higher cut-score. It was just like they’d moved the goalpost further down the field, one Buffalo educator (and apparent football fan) explained. More kids failed because they graded the tests harder.
But a lot more happened than that.
As State Education Commissioner David Steiner explained at the state’s press conference, the state tests have not simply become too easy. They have become bad tests. More »
In a recent profile in New York Magazine, charter school CEO Eva Moskowitz proclaims herself the savior of public education. However, the article makes clear that Moskowitz does not truly offer any solutions to the thorny problems of urban schools; instead, the culture she has implemented as CEO of Harlem Success has actually magnified problems. The gap between rhetoric and reality calls into question what Moskowitz’s real “mission” really is — and at $400K a year, that’s an important question to ask.
Though she makes the absurd claim in the article that her mission to change public education started as early as first grade (while most of us were concerned about which cartoon lunchbox we would get), Moskowitz’s real mission is to increase her own political power. During her early 2000’s tenure on City Council, Moskowitz conducted a series of education oversight hearings. (Which, according to the article, satisfied her childhood “Watergate” fetish.) In many respects, she intended these hearings to be a launching pad to higher office — but the plan backfired, as her coarse personality turned off voters and resulted in a nine point loss in the race for Manhattan borough president. Her current resurgence of interest in educational issues is intended as a pathway back into the public light, and perhaps higher office.
Moskowitz’s contradictory views on standardized testing are one hint that her interest in public schooling is more about playing to political rhetoric than thinking about what urban students really need to succeed. More »
Though local newspapers did not bother to ask them, any teacher could have named a key reason why state math scores are soaring while the federal TUDA for NYC is largely flat. In spite of their own best professional judgment, their complaints, and their protests, teachers in New York City have been compelled to teach narrowly to a narrow state test, and to use test results to determine what to teach. Teach to a test — and worse, teach to a bad test — and you can’t expect kids to know very much. In fact, you can’t expect them to get much of an education at all, beyond the education that is politically convenient for some and gratifies the ideological enthusiasm of others. NAEP asks for more of education, and that’s the more we are denied from giving them in NYC.
The ideology to which I am referring of course is the penchant everywhere to replace real learning with a spreadsheet education: education by the numbers sliced and diced and then sliced and diced again. More »
Caroline Hoxby’s updated report on New York City’s charter schools uses a provocative construct: she finds that Harlem’s charter students are making standardized test score gains that put them on track to substantially close their achievement gap with Scarsdale.
Hoxby, a Hoover Institution fellow and Stanford professor who has published extensively on charter schools (favorably) and teacher unions (unfavorably), looked at students who won admittance by lottery to certain New York City charters and compared their performance to students who applied but were not admitted.
How much do collaboration, mutual respect, and other aspects of the school environment matter for improving outcomes for middle school kids?
Quite a lot, apparently, and it shows in New York City’s data. Every year, the DOE surveys teachers, secondary students, and parents to find out what the school looks like to the people who actually spend time there. Basically, the survey asks whether the school community is a welcoming one that holds high standards, inspires kids to learn, and cultivates collaborative culture. The city tabulates the results, combines them with attendance data and then converts that to a letter grade. In addition, the DOE gives schools a separate letter grade that is based on student progress on state exams from one year to the next.*
Compare those two letter grades — for environment and progress — and what you will find is that environment matters. More »
The mayor has announced that he is expanding his plan for ending social promotion. The problem with that plan isn’t the goal, but rather the means by which to reach it: by relying (can you guess?) on how well the student does on state exams. Over-reliance on test scores for high stakes decisions is never a good idea, but relying on them for decisions about social promotion seems especially ill-advised. Students must attain a Level 2 to be promoted, but as the Daily News pointed out students can reach that standard just by guessing. And, on Thursday, Diane Ravitch had this to say: More »