And My Job Quality is Based on These Tests?! (Updated)

By heather on May 28, 2010

So clearly we’ve all been thinking a lot about the necessity of test scores in making high stakes decisions. I mean, test scores seem to be used in everything these days: teacher evaluations, a student’s college or career readiness, merit pay, even neighborhood real estate, you name it.

And, sure, there are test scores of sorts used in any number of other professions. My father used to come home talking about game show Nielson Ratings, my brother in-law looks to see the totals for his opening weekends. (Can you tell I come from a family of entertainment?) But somehow our test scores are different.

Our test scores reflect far more than our efforts and performance. They reflect how much sleep a kid got the night before. They reflect the recent divorce, the boyfriend’s breakup during passing period, the number of days the kid wasn’t at school, apathy, yesterday’s enrollment into the school, and yes, content knowledge.

But test scores are the American way, a game to those who succeed in them, aren’t they? And like any competitive sport, there is the “thrill of victory and the agony of defeat.” And, after all, we can’t all be winners, right?

But that’s just why teachers and schools have begun to circle their wagons and arm themselves with their voices loud against this threat of test scores running the show. Students should not begin their lives in the agony of defeat. We should be equipping them with what it takes to be victorious.

I think the reason why politicians tend to favor the need for competition in school is because it’s a language that’s worked for them, so they are confused about schools’ lack of buy-in. And while I’m all fine with a good healthy dose of financial incentive, we cannot compete unless we are all given the same resources. I mean, in the Olympics, does any swimmer in an antiquated swimsuit really stand a chance against someone decked out in the newfangled sharkskin suit? So is it for the inequity in school funding.

So clearly I’ve been thinking a lot about the fact that soon our jobs may be identified and retained in large part to my students’ ability to take standardized tests, a variable which, in my opinion, is only one step up from hire date as a means to retain a position (see my article for Teacher Magazine, “Does Last Hired, First Fired Really Make Sense”). And having just ended our own standardized testing, that good ‘ole CST, I am reminded yet again that one of those factors that affect achievement is the lack of quality of the tests themselves.

So as my students bubbled away earlier this May , I looked at the test booklet to get an idea of what the testing gods felt were important enough to assess this year. The quality of what I saw was truly tragic, and I thought I’d share a little of the asinine quality of these tests as a means to answer the question:

How can my job quality be based on THESE tests?

My recollection of the questions (we aren’t allowed to write them down from the book) is also compounded by the feedback from the students. But just to protect the sanctity of the actual questions which are top secret and must not be discussed at all costs, I’ve replaced all the actual terms with similar ones that hopefully get my point across.) Here are the kinds of questions we observed:

1. They were randomly asked to define the word “yachting” (remember, the actual word is disguised to protect its true identity for fear of offending the original word.) Now my Title I minority students (the majority of my school) had never encountered that word. And I was proctoring the advanced math group. You know, those kids who started Algebra as zygotes? Now, I’m not knocking the students. I’m knocking the test makers who clearly can’t seem to avoid culturally elitist questions.

2. The informational reading selections were, how do I say it? Dated. One was on reading the instructions on how to use an old crank Phonograph.

3. How ’bout the fact that there is a percentage of questions on the test which will be dumped if over a certain percent of students get it right? The fact is that the test makers assume that if a high majority of the students get the question right, (reflecting of course that a high majority of teachers actually taught that standard well) the question is trashed as being too easy.

4. How ’bout the fact that there are questions meant to just be “piloted” during the actual test, with the intention of being too hard for most kids? How does that make a kid feel while testing? Why are they using high-stakes tests to assess the quality of the questions on the tests? How nice that the test makers get to use the tests formatively, but the schools don’t.

5. I also couldn’t help but notice that there was at least one question which asked students to pick a synonym for a word, let’s say it was “brick,” and the choices to choose from weren’t even nouns like giving them “run”, “jump”, “laugh”, “cry.”

Assessments are meant to reflect what has been taught, not how to out-think a tricky question. That is not critical thinking. But in this day and age, tests are actually driving the curriculum itself. That being the case, why can’t our standardized assessments at least reflect the lessons we know are the ones that truly need be taught? (See my recent post here on College and Career Readiness in Assessments.)

If the tail must wag the dog, why can’t the tail at least be well informed about what might be knocked off the table with each swing? If tests have become instructional guides in their own right, should they not at least be good ones?

And most importantly for the topic of this post: if tests are to judge my performance as a teacher, or the quality of my students and their community, should they not at least ask questions that are applicable? There are many reasons why tests are not great ways to evaluate performance. Yes, there are students not putting in the effort or families not doing what they need to help students achieve. But it is also the quality of the very tests that is also setting up our teachers and students for failure.

Bottom line is this: good test scores does not a good teacher make, just as bad test scores does not a bad teacher make.

Care to share any of the bizarre or poorly constructed questions that you saw on the tests this year? Please share below.

____________________________________________________________________________________

And here’s a little Test Score blogroll so you can follow this topic from other edubloggers who comprise our Fellowship of the Ning (Otherwise known as Those-who-spoke-with-Arne-Duncan). Throughout June we will all be involved in discussions and webinars focused on the issues we raise in our series of posts on this topic. Remember, policy affects our practice. And all our voices need to be at the table. Check these out for further reading. Feel free to comment and participate on any of our sites:

Marsha Ratzel – “Reflections of a Techie”

Renee Moore – “TeachMoore”

Mary Tedrow – “Leaving No Multiple Choice Footprint Behind”

Anthony Cody – “Summer of Discontent”

Teacher Letters to Obama –

http://www.facebook.com/group.php?gid=166176941518&v=app_2373072738&ref=ts