With this year’s introduction of Common Core-aligned tests, flawed as they were, the city schools enter a new era. The transition will be a game changer that will bring angry reactions by teachers and students, and wider class and racial performance gaps. Student achievement measurement may become discredited for awhile, as an exasperated public throws up its hands in confusion.
Those could be the best things to happen to standardized testing in 10 years.
Achievement plummeted on this first try at new tests. City students scored 20 points lower in ELA and 30 points lower in math, with less than 30 percent of Grade 3-8 students meeting proficiency standards. But the tests set an extremely high bar — probably too high — in what amounted to a premature effort to test students against the new Common Core. Curriculum didn’t start to be available until late in the year and the Dept. of Education didn’t have the leadership required to manage such a dramatic transition.
But there is no going back. The New Common Core tests, which will continue phasing in over the next few years, may get better, especially if current state test-maker Pearson PLC moves out the way. But they will remain harder: they will ask students to do more explaining, analyzing and creating.
And here’s the thing: these are the very skills educators want to teach and have had to forego in favor of test prep. Right now, teachers are out of practice, and so are their students. But these are the skills they want to teach. So they will demand more autonomy, an end to the culture of test prep, more time and resources. As long as the state and city don’t slap ridiculous consequences onto the new scores, students and teachers alike will become less bored and hopefully more engaged.
Harder tests are going to result in widening gaps between better- and less-prepared students. Typically that means racial and wealth gaps, as well as gaps between English proficient and ELL students and between general and special education kids.
These were the gaps that No Child Left Behind set out to eliminate back in 2002. To the extent this succeeded, and it didn’t much, the cost was relentless test prep and/or dumbing down of tests.Now, as the gaps widen on the Common Core tests, parents will be outraged and politicians will distance themselves from the schools. So they should. Bringing poorly prepared students up to standards is the work of brilliant and passionate teaching, which has been forced underground in the NCLB era. Its reemergence can come only if good educators are free to work. They cannot be commandeered by mayors running numbers. A next generation teacher force can only be brought into being by experienced educators who are not ruthlessly tracked by narrow performance monitors.
Accountability and Legacy
The education mayor, the education president. These monikers turned out to be albatrosses around the necks of Michael Bloomberg, George Bush and many others. Their legacy is a culture of measurement, not of learning. Testing has become laden with consequences that the tests themselves were never meant to support, including judgments about schools, teachers, and even “where we are heading as a society.”
One of the best things these new tests could do is force accountability to grow up. The city has overwhelmed us with data that, on close examination, is really the same data points parsed a hundred different ways. What’s more, the numbers appear to lie, or at least, they zing up and down without apparent reason.
Common Core tests could do two things about accountability. The first is to force us to adopt a more rounded assessment of students and schools. The second is to put standardized tests back their rightful, and less overblown, place.
So less than a third of students meet standards. Well, what else do we know? How do students perform on social studies projects, lab work, art and music, sports, leadership activities, group tasks, or community service? What 21st century skills do they have; what ones need to be developed? What are the best models for teaching those skills? What can students tell us about what they do and don’t understand and what helps them learn? And how do we measure those?
There needs to be some opening up — more quantitative data that uses non-numerical measures. We have agreed that more than half of teacher evaluations will be based on observations of classroom performance. Why can’t we assess our students that way?
It would be a relief if tests become more the province of educators. Politicians don’t find scales, cut scores, p values and item analysis inherently sexy. But good measurement requires expert interpretation. If the heat gets turned down under testing, and we all agree it’s complicated, then public attention may return to subjects, to projects, to school activities and to learning.
Of course, there’s another scenario, in which the new tests are simply misused as the old ones were, to pass judgment based on partial evidence, to bash and shame and to claim undeserved legacies. But after a decade of this, teachers and parents, not to mention students, are pretty fed up. Their voices lend hope for a turnaround.
We all know that New York State created an evaluation system for New York City on June 1. And we also know that that there are three subcomponents in the new system: a state learning measure (worth 20 points) a local learning measure (worth 20 points) and an Other Measure, which, for ease, I’ll call observations (worth 60 points). We know also that the state laid out the broad framework for how learning measures and the new observation system will be implemented in our schools. But while teachers are becoming more familiar with the individual measures, they are far less familiar with how it all adds up once the year is done.
So let’s begin at the end with the cut scores the commissioner imposed. Cut scores are the numerical breaking points between levels (between Developing and Effective, for example). There are cut scores within each subcomponent as well as within the range for the final rating, which goes from 0 to 100. The subcomponent cut scores for NYC are different – and more favorable – than the cut scores the rest of the state has to use. First, let’s look at NYC’s cuts.
For NYC, the state has assigned a certain number of points to each level of performance in all three subcomponents. Thus, a teacher who is Effective in all three subcomponents receives at least receives 15 points for the state (Comparable) measure, 15 points for the local measure, and 45 for the observation measure. This teacher would wind up with an overall score of at least 75, and be considered Effective overall. If that same teacher had received, say, 53 points for observations, but fewer points on the learning measures (for example, an Ineffective/11 and a Developing/14), the overall score would still be Effective (78).
So, you may wonder, just how do teachers earn their 11, or 15, or 50 points? The short – but very important – answer here is that they do not earn them, or at least not directly. Rather, each subcomponent has its own way of expressing a meaningful result, and that expression has nothing to do with 0-20 or 0-60 score ranges. An observation result might show which of the four levels of the Danielson rubric best reflects your teaching. A learning result might indicate the percentage of your students who met their targets. Those different kinds of results from different aspects of teaching have to be converted into a common language, and that language is the 0-100 scale.
Comparing NYC’s Cut Scores To the Rest of the State
Understanding the scores as conversions from something else is crucial, especially in light of some blog posts that surfaced last week, wherein the writers – after comparing the NYC-imposed cut scores to the cut scores mandated throughout the state – came to the perfectly understandable, but entirely erroneous conclusion that NYC teachers were thoroughly and completely screwed.
The reality is just the opposite. The cut scores in force throughout the rest of the state are the problem; the NYC cuts are actually the fix.
So let’s compare. First, here are the statewide cuts. Note in particular the circled number.
And here, again, are our NYC scoring ranges:
As the circle indicates, a teacher in NYC who is rated as Developing receives at least 13 points for the state measure. The same is true for the local measure.
Anywhere else in the state, however, that same teacher would only receive 3 points.
In other words, teachers in New York City earn 10 more points than do teachers elsewhere for the same level of performance (that is, for the first rung of Developing).
How the Conversions Work
Of course, if you don’t understand that the 0-100 scale is a conversion, you might get a little freaked out, and that’s what happened with a lot of bloggers. What they wanted to know was why NYC teachers have to earn a whole 13 points just to get past Ineffective, when everywhere else in the state, they have to earn just three. It’s an understandable question but, again, teachers don’t earn points; rather, the results they earn can be converted into either 13 points (in our world) or only three (in theirs).
So, let’s say that a teacher’s learning results are based on the percentage of students who meet their learning targets. Many districts use such a method for determining learning measures, and NYC will be one of them. Other districts can negotiate exactly what constitutes performance at the first level of Developing, but no matter what they negotiate, the conversion remains at three points. Let’s compare NYC’s conversion to that of the four so-called model districts that use the same system:
…then the teacher is…
…which converts to..
of students meet their target
of students meet their target
of students meet their target
of students meet their target
of students meet their target
In these districts, teachers will be rated as Developing if somewhere between 50 % and 80% of their students meet targets. In NYC, teachers with similar success get more points in the conversion.
Why the Special Scoring Bands?
So why was the state so generous with NYC? Actually, it’s got nothing to do with generosity. Like I said before, these cut scores are a fix.
You’ll see this if you look once more at the statewide cut scores, and particularly at the composite scores. A teacher needs at least 65 points in order to be rated as Developing. So look at what happens to a teacher who is considered Developing in all three subcomponents. Since teachers who are considered Developing in both learning measures can be assigned as few as six points (three in each) in the statewide cut scores, a teacher who is also Developing in the observation measure would need to be assigned 59 points (because 65 minus 6 equals 59) in order to be rated Developing overall.
In other words, teachers rated as Developing in all three measures would wind up as Ineffective overall if they received fewer than 59 of the 60 points in the observation measure, just because of a scoring anomaly. Districts could negotiate whatever they wanted for the 60 points, but 59 wasn’t possible. After all, if 59 points get assigned to a Developing teacher, how many points would go to teachers who were Highly Effective? But on the other hand, if the district dropped Developing to, say, 58? Then some Developing teachers would get only 64 points overall (58 plus six), and be labeled Ineffective.
And the problem wasn’t just at that Developing cut – it showed up throughout the scoring ranges. Ultimately, depending on the observation bands the districts used, teachers rated DDD in the subcomponents – or even EDD – could potentially find themselves thrown into the category of Ineffective overall, unless the cuts were fixed.
For smaller districts that might not matter, since the vast majority of scores would add up just fine, so long as other districts were smart about where they set the cuts. But in a city the size of NYC, hundreds or even thousands of teachers may have been labeled “Ineffective” regardless of their substantive results. The actual number would be impossible to predict, but in anticipation of that problem, the state gave us cut scores that guaranteed that the meaning of the Final Rating would reflect the meaning of the subcomponents from which it was derived.
One last question remains: Why did the state set the problematic cut scores for the rest of the state in the first place? Basically, the state wanted cuts that would make it impossible for teachers who were Ineffective in two different learning measures to overcome that rating, even if that teacher received all 60 of the observation points. The statewide cuts do that, but they also create the potential for many additional Ineffective that simply can’t be justified in any system. For NYC, all of those additional Ineffectives have been eliminated, but the state has still included language to the effect that a teacher who was rated as Ineffective both learning measures must be considered Ineffective overall, regardless of the score. That is an outcome that would have affected us in the previous system and would affect us in this one as well.
I started the previous paragraph saying that one question remains, but of course that is laughable. I know that this post is a far way from covering all the questions teachers have about the new evaluation system. Future posts are on their way.
But whatever pros and cons we may find in other aspects of the new evaluation, the final score cuts are an improvement. It was a change that only the state could make – and it did.
10 percent of the schools produce nearly half the college-ready graduates
Last week the city announced that 22.2% of students from the high school Class of 2012 met the state’s college-ready standard, up from 21.1% for the Class of 2011. What the announcement didn’t say was that this already weak college-readiness rate was inflated by a small group of schools that contribute a disproportionate number of students to the city’s college-ready percentage.
The differences between schools were so great that the city’s overall college-readiness rate of 22.2% did not represent the reality for even most city schools. In fact, only a quarter of the city’s high schools had a college-ready rate that was 22% or better.
Here’s one way to look at the numbers:
Out of 352 total schools for which data are available, the top 35 schools — 10% of the total – graduated nearly half of the city’s 16,600 total of college-ready students, boosting the city’s overall college-ready figure and obscuring the lower rates achieved by the overwhelming majority of remaining schools. As the chart below shows, at these schools 73% of the graduating cohort is college-ready compared to only 16% for the bottom schools. The 35 schools that skewed results include the likes of Stuyvesant, Bronx High School of Science, Brooklyn Tech and Townsend Harris as well as a few neighborhood schools like Francis Lewis High School and Midwood High School.
The distortion shows up most dramatically when you split the schools in half. The top 50% of schools contributed almost all the students — 15,600 students — to the city’s college-ready total of 16,668. The 170 schools in the bottom half of the rankings, though they have an estimated 22,500 students in their senior cohort, contributed a total of only 1,035 pupils to the total college-ready ranks. The average college-ready rate for this group was less than 5%.
The huge differences in college readiness by school means that the standard mechanism for calculating the city’s rate doesn’t tell us much about the reality of New York City high schools. Many of the top schools have a very stable college-ready rate and have been in the top 10% of schools for the past three years.
When the DOE reports on the city’s college-ready rate, it should take these things into consideration and report in the most transparent way. For 2012, this could have been accomplished if the DOE had said the following:
While the city’s overall college-ready rate is 22.2%, only 25% of the city’s high schools achieved this rate.
The city’s college-ready rate of 22.2% drops to 16% if the top 10 percent of schools for college-ready students (approximately 35 schools) are excluded from the calculation.
If you analyze the results by dividing the 352 schools in half by college-readiness rates, the difference becomes even more marked. The overwhelming number of college-ready students come from the top 50% of schools in the college-readiness rankings. The schools in the bottom half of the rankings manage to produce only about 1,000 college-ready students – less than 5% of the system’s college-ready total.
The college-readiness rate was created by the NYS Education Department to identify high school students who have graduated and who are academically ready for college-level math and English courses. To be deemed college ready, a student must pass the NYS Math and English Regents with an 80 and 75 or better, respectively. This benchmark was set based on the experience that the City University of New York (CUNY) had with students who attended NYC public high schools. Students who fail to meet the standard are required to enroll in remedial math and English courses or pass special exams that allow them to test out of remedial courses
There were 406 high schools with students in the 2012 graduating class. Data for 54 of these schools, however, was not published. The DOE withheld the information on these schools in order to comply with the Family Educational Rights and Privacy Act.
When the mayor announced the Class of 2012 graduation rates on June 17, he blamed higher Regents requirements for a drop in the percentage of on-time graduates. But this is not the full explanation. There were several disturbing elements in the numbers that could signal a long-term flattening of graduation rates.
The drop in the city’s graduation rate was small — less than a percentage point — at 64.7 percent for the Class of 2012 versus 65.5 percent for the Class of 2011 (including August graduates). But it was unexpected. This is the first time since 2002 that grad rates have dropped.
It couldn’t be the Regents alone. The higher Regents requirements are not new. They have been phased in since 2005, when the entering high school class had to pass two of its five Regents with at least a 65, instead of 55. The following three cohorts each added another Regents at 65 — and graduation rates continued to climb.
The last test was not the hardest. There was no particular Regents test saved for the final phase-in year. Passing algebra, ELA, global history, US history and a science in any order was fine.
The decline was confined mostly to New York City. Using the state categories, high-need rural districts, large suburban districts, “average need” and “low need” districts all had increasing graduation rates for 2012. Though rates in the “Big Four” urban districts of Syracuse, Yonkers, Rochester and Buffalo dropped (only two of them as much as New York City). still, the overall June graduation average for all districts outside of New York City rose to 81.6 percent, up from 81.2 percent the previous year.
The students who are traditionally the hardest to graduate fell further behind. The standout was English language learners. Just 40.5 percent of city ELLs in the Class of 2012 graduated by August, a drop of almost five percentage points from 45.1 percent for 2011 and 46.1 percent for 2010. This cannot be only about the Regents ELA. That test is often students’ first or second Regents, and of course can be taken multiple times. In addition, former ELLs — those who began high school as ELLs but tested out of the classification — have also seen declining graduation rates for each of the last three years.
Black and Hispanic students made no progress. The on-time graduation gap between black and Hispanic students and their white and Asian counterparts was unchanged again: 20 points between blacks and whites and 22 points between Hispanics and whites. They have been stuck at these levels for the last five years. Students with disabilities had a graduation rate of 27.6, virtually unchanged for the last two years and far below the 45 percent statewide average. If black and Hispanic students, who make up more than 70 percent of the city’s students, cannot narrow the gaps between their rates of graduation and those of whites and Asians, then the overall grad rate is unlikely to improve.
Finally, there is the college-ready graduation rate. Just over one-fifth (21.9 percent) of the entering class in 2008 was prepared to succeed (minimally) in college or career by June 2012, judged by Regents scores. That’s a point higher than last year, but it’s a sign of profoundly mediocre education, with all this mayor’s reforms. The black college-ready rate was 11.1 percent; for Hispanics it was 12.2 percent.
Our graduates are regularly stuck in remedial courses when they go to college — in fact the number has been increasing. As Yoav Gonen in the NY Postreported, since 2009, the remediation rate for city public-school graduates who enroll at CUNY 2-year colleges has increased by about 6 percentage points — from just under 74 percent to fully 80 percent last year.
The numbers suggest the city is hitting a wall. Graduation rates did climb encouragingly for several years, as teachers worked extraordinarily hard to meet draconian test-based accountability measures. But that takes the system only so far. College readiness involves abilities that a compliance and test-driven education cannot provide.
More of the same won’t do it. Next year high school graduates must pass a Common Core-aligned Algebra Regents, and further Common Core tests will be introduced from there. Those tests are harder. There were already whispers about heavy use of credit recovery and dumbed-down Regents tests in getting the graduation rate this high. Now it’s hard to see where the push for better rates will come from.
A high school diploma is a valuable piece of paper. It may not be sufficient, but it is absolutely necessary. Reaching an acceptable four-year graduation rate, say, matching the “average need” districts’ 85 percent, requires a sea change. It starts with an education mayor, but a real one this time.
Were they the good old days or the bad old days? Back then, if a kid wanted to hang in the street, flop at home, or binge in some way, all he had to do was break a school rule to get suspended from the building for five days, maybe more. It was an “out of sight, out of mind” deal in which teacher and student got a hiatus from each other, equivalent to (though lacking moral equivalency) a vacation.
Then came the Alternate Learning Centers (ALCs). Suddenly, educational deprivation was no longer a viable tool for attitudinal correction. And the notion of building kids’ characters by temporarily starving them of communal learning opportunity began to go the way of Twinkies.
There are almost 40 ALCs citywide, located in all boroughs. They are longer or shorter-term suspension sites, depending on the cause of their suspension. These sites serve students who are on superintendent suspensions, not principal suspensions.
Superintendent suspensions for 2011-2012 declined by 12 percent from the previous year to 13,258. That represents just under one-fourth the number of principal suspensions for the same year.
Some ALCs have their own location; others share a school building, though their student populations don’t mingle. All of them are serious places for teaching and learning. Expectations are enforced with kindness and firmness. The atmosphere is not punitive, though no unreasonable excuses or “getting over” on authority are tolerated.
The students have been suspended but there is no suspension of the continuity of instruction. ALCs are not warehouses or receptacles for so-called “problem kids.” Regardless of a student’s length of stay and his age or academic level, he’s got a variety of curriculum-based material to study. More »
New schools find themselves more likely to be identified as Focus schools on the state’s new Focus list.
A lot of middle schools have closed these past ten years in New York City and a whole lot more have opened up. In fact, nearly a third of our middle schools are new, and when we look at the higher need schools, the proportion jumps to 43%. These new schools opened as part of a reform agenda whose ideas are simple and familiar to all of us: empower principals to pick their staff and then focus “like a laser” on achievement. Hold the staff accountable; threaten schools with closure if they fail. And if you do that, the theory goes, they will be far more likely to succeed.
So how’s that working out?
Apparently not very well. The state has designated 100 middle schools in NYC as Focus schools. New schools are over-represented:
One third of the middle schools are new schools, but they represent 40% of the Focus list.
43% of the higher need middle schools are new, but they represent 58% of the higher-need schools on the Focus list
Details follow, but what this means is that, at least according to the state, once new schools reach high concentrations of high-need kids, they are overwhelmed by the same challenges as the old schools down the block. More »
For years, Bloomberg’s high school admission policies have been concentrating the city’s most at-risk students in certain schools. What with complex, market-driven enrollment policies on the one hand (which favor the families best equipped to negotiate the system), and high-stakes accountability systems on the other (which reward schools that teacher fewer at-risk kids), students have been disenfranchised by Bloomberg’s policies.
The UFT and others (see here, here, and here) have been pointing this out for years, and for just as long, the DOE has denied it. But now it turns out that even as Bloomberg makes his denials, he and the DOE have been scrambling for cover. NYS Education Commissioner John King has put on the pressure, and in May, the DOE sent him a letter claiming they would address the problem, noting that “concerns about situations where our choice-based system may be leading to an over-concentration of students with disabilities, English language learners and/or students that are performing below proficiency in certain schools.”
In case you have not been paying attention, the mayor is vowing to dismantle about 30 school communities for reasons that pretty much no one can figure out. In fact, things have reached such a level of absurdity in New York that there are very few New Yorkers who actually believe that the campaign against our schools has anything to do with “school quality” or a desire to make things better for at-risk kids.
I mention this because in the face of such a situation, it seems ridiculous for me to continue my private crusade to correct the DOE’s misrepresentations about the schools that close and the new ones that rise up in their midst. The DOE mask, truly, is off. Still, even though no one in New York believes the mayor, New York City mayors often have national ambitions. It can’t hurt to set the record straight.
So, let’s look at a few big old schools and the new ones that replaced them in the same building. In particular let’s look at the schools’ comparative reading levels and comparative math. More »
Last week, Leo Casey gave Edwize readers the real story of Pascal Mauclair, whom the NY Post declared was the “at the bottom of the heap” when the DOE released the Teacher Data Reports to the press. The DOE gave Ms. Mauclair a “0” on her report, but the results seemed, to put it mildly, arbitrary. As Casey pointed out, Ms. Mauclair was graded on a small number (11) of high-need (ESL) students who were compared to other students learning in very different, departmentalized, classrooms. Aside from that, Ms. Mauclair has a reputation as an excellent teacher. As her principal said, “I would put my own child in her class.”
All this alone should be enough to clear Ms. Mauclair’s name. But this week fresh evidence shows that Ms. Mauclair’s report should be declared invalid altogether by the DOE.
In order to understand the problem with Ms. Mauclair’s report, you need to understand the important role of other-subject tests in the value-added formulas. When we think of a math report we think of the progress a teacher’s students make in math from the prior year to the current year, weighted for various factors, or variables, such as gender, poverty, attendance, and so on. But what we may not realize is that the most determinative variable is the student’s score from the other subject. In the case of a math report, for example, the prior year’s ELA score is generally considered a better predictor of the students’ next math score than are poverty, gender, or any of the other variables that the formulas factor in. More »
This morning, Mayor Michael Bloomberg and Chancellor Dennis Walcott appeared at an American University forum with U.S. Secretary of Education Arne Duncan, and the mayors and school superintendents from Chicago and Los Angeles. A video of the forum is here.
Bloomberg defended giving out invalid and inaccurate Teacher Data Reports as providing “information” to parents, saying it was “arrogance” to suggest that peddling wildly inaccurate information was a bad idea.
He made a feeble attempt at backtracking from a prior statement that in his ideal world, he would fire half of the teachers and double class size. “Class size is important,” he opined, but not as important as other things such as teacher quality.
At the very end of the program, Bloomberg displayed his education acumen and keen political ear by declaring that “teaching to the test is exactly what we should do.” And in defense of this position, he invoked a Pete Seeger song, “Waist Deep In The Big Muddy.”
While Michael Bloomberg dedicated his youth to obtaining a Harvard Business School M.B.A., the best minds and hearts of his generation were fighting to end the war in Vietnam. With very little knowledge of that struggle, Bloomberg did not seem to realize that Seeger’s anthem (lyrics below) is actually a metaphor for the American political leaders who mired our nation into the Vietnam war, at every turn escalating the war and bringing us in deeper. Indeed, it is a metaphor that has considerable resonance for the “education reform” work of the NYC DoE under his administration. (Video and lyrics after the jump.) More »
[Editor's note: The author is a teacher at PS 122 in Queens.]
The recent release of NYC’s Teacher Data Reports has stirred up a wide range of responses from all of the relevant stakeholders in our city’s school system. As a teacher whose name was published in the local media with a corresponding characterization of “Below Average,” I am upset, angry, even demoralized. After a great deal of personal reflection, I felt compelled to reach out to fellow teachers and, especially, to the parents of the students I teach.
For me, it is important for people to know that I teach in the same school that I attended as a child; it is the same school that both of my siblings went to as well. As three children of immigrant parents, we owe a debt of gratitude to our alma mater, and I strongly feel that the experiences that we had at PS 122 were instrumental in paving the way to a life of higher education. Needless to say, I would do anything for my school.
Furthermore, it is critical for one to understand that all of my students meet the standards promulgated by New York State, and all but a small few actually exceed those standards. And yet, even though my students consistently outperform the vast majority of their peers throughout the city and state, the Teacher Data Report concludes that I am an ineffective teacher. More »
The New York City Department of Education is the largest public school system in the United States, serving about 1.1 million students in nearly 1,500 schools. It employs about 80,000 teachers. Joel I. Klein, the current chancellor, was appointed by Mayor Michael R. Bloomberg in July 2002.
Real stories from the classrooms of new NYC public school teachers. Take a look.