Tuesday, September 18, 2007

An Assessment Tale

I'm frustrated by assessment. Here's the thing, ten years ago as a new faculty member, I heard the first I'd ever heard about assessment. Never heard the word in grad school, but my first department meeting, I heard:

From the assessment head:
Portfolios! We're going to assess student learning using portfolios, and then we'll be able to allocate resources based on what we learn about student learning! We've ready this great book by an assessment guru, and s/he says portfolios are ideal! Every school out there is using portfolios! If there's a problem in an area, we'll figure out why and that will help us know how to allocate resources!

Me (as bright eyed and bushy tailed as someone can be after a disertation and the MLA meat market):
Okay! Portfolios! So, I need to set up some assignments in my class so they overtly address the portfolio rubrics. And then I need to set aside time in my class to help students put stuff in their portfolios, and that means I have to set aside time in my class to introduce students to the whole portfolio idea and try to convince them it's meaningful to them! I'm on it! I'm a new faculty member and I'm going to try do my best!

New job a few years later, assessment head:
We use portfolios! I read a book by this guru and portfolios will solve all our problems! And look, we have a cool new electronic way to do portfolios! And we'll use the portfolio assessment to help us allocate resources! This is the best thing since Fry Bread!

Me (a little less bright eyed after another go at the MLA meat market):
Okay! Portfolios! I've heard of these. I'll set up assignments to address the portfolio rubric directly, and set up class time to introduce students to the concept of a portfolio and help them put things in the electronic form. Because you all must know what you're talking about, and there are resources involved, right?

Ten years later:

Assessment head:
Portfolios suck! We've got ten years of data, but faculty members didn't buy in. I just read a new book and there's a whole new way to do assessment! It will only mean a little more new work for you in your classes. And some more committee work. But we'll tie resources to the results within a few more years.

Me (perhaps a bit cynical?):
We know from the data that we're unsatisfied with student writing. We know from the data that our class sizes for writing classes are nearly double what they should be for ideal student learning. How about putting some resources in now?

What makes you think you know what you're doing this time? Why should we trust this guru when the last one was full of excrement?

Unfortunately, I was saying just that as I vented in a friend's office doorway when the assessment head came and went into the next office.

There are NO RESOURCES to put behind changing things to make student writing better. We've been through this, heard it. Quit lying. Get yourself some ethics and be honest. Actually do some serious research before you pass off your latest guru's work as meaningful or valuable.

The first time, I bought in, worked hard to help students learn about the portfolio stuff, to have assignments that were appropriate for the rubrics, and to encourage students to participate. And what's the result? I wasted my time and energy. And now you want me to waste more time and energy on a new idea that hasn't been tried out and for which you have no evidence, really?

And you want me to buy some prime farmland where?


  1. That's always the rub: the assessment gurus aren't working with the budget gurus to create either the time or resources to create change.

    This sounds a lot like conversations I've had around here over time, too. Except our campus guru is pushing e-portfolios now.

  2. Yes, big plans, no resources. Where I am we're being threatened by SLOs.

  3. And, they wonder why the buy-in rate is so low?

    I love the fact that they'll run with the ideas of one guru while ignoring multiple studies on the relationship between class size and writing ability.

    I suspect that someone wanted to either visit pals someplace where they do portfolios, or there was a cool conference to go to someplace warm, in February...

  4. I'm glad the new assessment guru hear d you. You're assuming, though, that this is really about making student writing better, which (as you correctly pointed out) would be a possibility if they'd spend more money on what really works (lowering class sizes).

    Maybe this is too cynical, but when I hear this "Here's the new shiny thing!" speech, I hear this from the guru: "It's not sexy to do what works if it costs money, and it's not a line on the 'goals and achievements' summary that'll get me a bonus if I'm not on the bandwagon with the next new thing." Funny thing about that: the gurus come and go, and they rarely end up actually teaching in the environment that they promote so enthusiastically.