Log in  |  Search

Test Scores, Multiple Measures, and Circular Reasoning

Anthony Cody’s recent reflection on this year’s Education Nation program on MSNBC offers an important caution to those trying to develop “multiple measures” for student learning and effective teaching. If the decision to use a given measure is determined solely by whether or not it’s linked to higher standardized test scores (as with the Gates Foundation’s Measures of Effective Teaching study), then you don’t really have “multiple measures.”

Tracking test scores can be an important tool in helping students make progress, and it is useful to know which elements of classroom practice have a significant impact on students’ performance on end-of-the-year tests. For example, teachers in Chicago who had high ratings on Charlotte Danielson’s framework for evaluating effective teaching have also been shown to have higher value-added scores. However, when test scores are used as the sole measure of effective teaching and learning — or when valuable aspects of effective teaching and important types of student learning are discarded or ignored because they don’t align with standardized test results — our students are the ones who ultimately pay the price.

Do you notice what is bothering me? Mrs. Gates begins by acknowledging that good teaching cannot be reduced to a test score — or at least that this is often said. She then asserts that the half billion dollars they have spent on research in this area have uncovered a number of things that can be measured that allow us to predict which teachers will have the highest test scores. A great teacher is defined over and over again as one who made sure students “learned the material at the end of the year.”

[…]

In spite of all the billions they have spent, it appears that the Gates Foundation is laboring under the same logical fallacy that doomed No Child Left Behind. In a way which employs circular reasoning, they have defined great teaching as that which results in the most gains on end of year tests, and then spent millions of dollars identifying indicators of teaching that will yield the best scores.

The most deceptive strategy is how they then try to pretend that these indicators are “multiple measures” of good teaching. In fact, these are simply indicators of teaching practices associated with higher test scores. In spite of Mrs. Gates’ feint at the opening of her response, everything she describes, all these things that supposedly go beyond test scores — peer observations, student perceptions — are only deemed valid insofar as they are correlated with higher test scores.

Melinda Gates begins with the question “How do we know a teacher’s making a difference in a student’s life?” That is an excellent and complex question. However, when we look at her answer, we find she commits the logical fallacy known as “begging the question.” One begs the question when one assumes something is true, when that is actually a part of what must be proven.

The question she begs is “what defines great teaching?” This is not answered by finding teaching methods associated with higher test scores. This question remains hanging over the entire school reform enterprise. Until we answer that question, we are devising complex mechanisms to elevate test scores assuming this will improve students’ lives, when this is manifestly unproven. In fact, I would argue that many of the strategies used to boost scores are actually harmful to our students. And many dimensions of great teaching are not reflected in test scores — and are systematically undermined when educators are held ever more “accountable” for these scores.

Print