Strategy and Analysis to Defend and Transform Public Education

Garbage Science: Teacher Evaluation by Test Scores and Some Ideas for Stopping Them in Oakland

Some of you might have already seen the shocking results in the New York Times today.  Apparently, all the teacher evaluations programs pushed on school districts by Obama’s Race To The Top and the corporate de-formers have found a shocking conclusion: most teachers are, in fact, “highly effective” at their jobs.

Diane Ravitch does a great job of poking holes in this “realization” and cites some of their statistics:

In Florida, 97 percent of teachers were deemed effective or highly effective in the most recent evaluations. In Tennessee, 98 percent of teachers were judged to be “at expectations.”

In Michigan, 98 percent of teachers were rated effective or better.

This is serious news for Oakland.  As many of you hopefully know by now, GO Public Schools & Co. (including Youth Together, Youth Uprising, SEIU 1021, OCO, and Education Trust-West), is making a serious push to evaluate Oakland teachers by student test score data.  They are not alone.  Superintendent Smith is heading the same direction in conjunction with 8 other California school districts (in the group called California Office to Reform Education (CORE)).

Following President Obama and Education Secretary Arne Duncan’s lead they are making a full-court press to push through these evaluations in line with the Smarter Balanced Assessments Consortium (one of 2 RTTT-funded consortia developing “next-generation” assessments of students and teachers).   According to CORE, these new evaluation policies will be “designed” by Spring 2014, piloted during 2014-15, and fully implemented in 2015-16 (see the last 5 slides of their powerpoint here).  What this means, as CORE recognizes, is that teacher unions will have to accept teacher evaluation by data–and very soon.

Which brings us full circle–to the total lack of scientific validity of teacher evaluations by student test scores.  Deborah Meier, an educator and education expert based in New York City, made this point very convincingly in a debate with Eric Hanushek, a Stanford economist and deform-ideologue (think global warming denying scientists).  She notes that when you put garbage data in, you get garbage data out.   Put another way, despite all the attempts to twist student test scores with fancy statistical tricks, students’ test scores still do a much better job of measuring things besides teacher ability.  They can measure inherent genetic ability, how hot the classroom is during test day, or how tired that student was.

What do test scores mostly measure? Poverty.

What do test scores mostly measure? Poverty.

Most importantly, however,  test scores mostly measure poverty.  Numerous studies have confirmed that test score data correlates to poverty.  This is clear to most Oakland teachers, parents, and students.  Hunger in the home, family unemployment or imprisonment, and living in blocks where street violence is all too common are problems that students do not leave at the school door.  Then when they step in the school, they find missing materials, new and inexperienced teachers, and crumbling facilities–again all related to poverty.

So in the end all these attempts by GO and their corporate sponsors distract social change from the real problems of poverty.  As we pointed out before these reforms basically mean nothing until we address the real underlying issues.

So what is to be done?

  1. Let’s put pressure on the coalition put together by GO to retract their proposals.  If you know folks in Youth Together, OCO, or any of the other partners, ask what are they doing blaming teachers and not working to solve the problems of poverty in Oakland?
  2. Let’s be ready for Tony Smith to push these evaluations on teachers.  We will need to be educated to resist their very powerful and highly funded think-tanks, pseudo-scientific arguments, and media connections.   Two upcoming School Board meetings will be discussing teacher evaluations on April 10 and April 24.   GO and Tony Smith’s agenda shouldn’t go unchallenged at these meetings.
  3. Relatedly, we need to start a truly progressive campaign that organizes to change the real problem affecting Oakland’s students: poverty.  We need to fight poverty in schools and out of schools.  This means demanding things like jobs for all and papers for immigrants outside of schools and smaller class sizes and fully funded education in schools.  Let’s unite teachers, parents, students, and community members for this.  And when we do that we’ll be able to bring those deeper relationships into the classroom and make even more gains there.  We can even build new and more relevant curriculum out of those struggles that will engage students and teachers even more, like teachers and students have in Oaxaca, Mexico.   When we start down this path, then we’ll really see our schools–and much more–improve.

4 comments on “Garbage Science: Teacher Evaluation by Test Scores and Some Ideas for Stopping Them in Oakland

  1. Pingback: Connecting the Dots: Bay Area Millionaires Buying School Board Elections Across the Country | Classroom Struggle

  2. Carlos Zambrano
    June 12, 2013

    These are great points. We should continue pretending that there is no possible way to objectively judge teacher quality (except for seniority and masters degrees of course).

    Is judging teacher performance based (in part) on test scores entirely fair? Nope.

    But guess what – real professions don’t work in an entirely fair manner. Lawyers are judged by how many cases they win, even if they did their best and the odds were stacked against them. Surgeons are evaluated based on how many of their patients live, even if they do their best and the odds are stacked against them. Hedge fund managers are judged by returns they produce even if (you guessed it), they did their best and the odds are stacked against them. That’s why those guys make the big bucks and teachers don’t. They’re willing to get out there and pursue results, instead of sitting on the sidelines whining about how the evaluative framework isn’t fair.

    Ravitch and her ilk want teachers to be paid and treated like real professionals. Fine. But that means accepting that results come first and the process won’t be wholly fair.

    • ClassroomStruggle
      July 15, 2013

      Carlos we appreciate the response even if we clearly don’t agree.

      First I want to summarize your response to see if I understand the content. You start by agreeing that judging teachers by tests is not totally fair but continue to say that all professions need some sort of “objective” measurement and that many other professions work under such conditions. Moreover these measurements are imposed on these professions even if “the odds are stacked against them” concluding that teachers should except the same systems because results come first.

      Let me start off by saying that we in ClassRoom Struggle and many many teachers are not at all against improved results. In fact, I would say that as teachers struggling in classrooms day to day we are some of the people most committed to seeing our children grow up to be educated, creative, mature, adaptive, and happy people. Now the important question to answer is: do standardized tests actually lead to improved results? To actually change results, a few things would need to happen. One, tests would need to be able to measure real learning and not a superficial measurement of remembering facts. Real learning looks like creative thinking, comprehension of different forms of media (written essays, articles, books, fiction and non-fiction, speeches, etc.), learning how to interact and work with other people to be effective and respectful, being able to articulate your own points of view and back these up with well-selected and integrated pieces of evidence, and many other skills. Two, the tests would need to actually be able to reflect back on the process of teaching and help it improve. It would need to provide real information that could help change the learning process for the better. Crucially this information would need to be worked into the learning process in a healthy and respectful way that can encourage the risk-taking, the time for reflection, the creative thinking, the interaction, etc. needed to actually improve what above I defined as real learning. In my mind these are some key elements of effective evaluations.

      Now do the current standardized tests handed down to us by corporate elements and politicians accomplish any of this? Absolutely not! Our current standardized tests regimes stultify this learning process. They don’t produce real learning results. And in fact by putting themselves on the pedestal of “objective” knowledge they redefine what is learning to be fill in the box, black or white, individualized, and uni-directional (that is not interactive). There are even corollaries in this process with your lawyer and doctor example. Say a lawyer is successful in getting a conviction for someone who possessed crack cocaine for their personal use and is addicted to it. By “objective” measurement the lawyer is successful. Now in the larger societal picture, will that result lead to this person being able to overcome their addiction? Or take the example of a surgeon who operates on a person successfully to remove a cancerous tumor in their stomach. But consider that many forms of cancer are caused by extremely unhealthy societal patterns of industrialized food, toxic environments, lack of exercise (often because of overwork and lack necessary facilities), etc. Again, did measuring that doctor by that one surgery measure that person’s healthiness? And did it actually lead to changes that would help them become healthy?

      I have written a lot. So I will end with saying that we at CS are not against evaluations if they are the right kind. The kind we have now are politicized and used by corporate interests to dumb down students. We want quality evaluations that actually help the learning process and acknowledge the learning and teaching we do is so much more complicated than filling in a box.

  3. Pingback: Good Teaching is an Art, Not a Mathematical Formula | Still Advocating

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: