On Assessment

Recently, Jeffrey Alan Johnson had an interesting article in Inside Higher Ed, "On Assessing Student Learning, Faculty are not the Enemy."  It's a smart article, and makes good points about how assessment isn't "neutral" and will require change in curricula.  Johnson notes that administrations using assessment to demand changes in curricula is a change in the power structure; rather than faculty controlling the curriculum, administration uses assessment to control it.  (Administrators may say they don't, but Johnson argues against that.)

Part of the problem Johnson notes comes in what gets assessed.  If we assess Pattern Regularity in Deepwater Basketweaving, and not Basket Shaping, then faculty folks are going to be forced to focus on pattern regularity, and the basket shaping parts of the curriculum are going to fall by the wayside. 

And the faculty may, given the mandate to choose to assess something that can be measured and proven, choose to assess pattern regularity, rather than basket shaping, just because it can be measured.  A regular pattern may mean nothing without a well-shaped basket, of course, but we can't assess every single thing we do, because we do a whole lot in any course, so what gets assessed is going to be what can be assessed most easily because it can be measured.

Business folks are all about how you have to be able to measure something in order to manage it.  That's easy if you're measuring how many widgets get produced, but considerably harder if you try to figure out the quality of different foods cooked in a restaurant, or even more, the education of a student.  If you measure food quality by taste, then that's one thing, but if the consumer gets horribly sick from food poisoning four or five hours later, then the food quality was bad no matter what it tasted like.  That won't make it onto the survey at the end of the dinner, though.  Education is even more complicated because many people won't really gain an appreciation of their education until they've been out of college for a couple of years or more.  And some students won't put things together fully until they've had a chance to mature a bit.  Let's face it, Jane Austen in your 20s is good; Jane Austen in your 30s is WAY better.  It's not Austen's quality that changed, it's the reader.

Johnson also talks about the dismissive ways administrators talk about faculty when they talk about assessment, as if faculty are recalcitrant children.

The comments to Johnson's article are also useful and telling.


Over at Confessions of a Community College Dean, the Dean reads Johnson's article and talks about how difficult assessment is to do when there's no obvious capstone (as in most community college programs, especially those intended for transfer).  One thing he talks about is that his college has an assessment committee made up of people whose workloads are adjusted so that the assessment work counts, and those people read assignments and make recommendations to their colleagues.


If you've read here for a while, you've probably gathered that I have a negative attitude towards assessment.  Reading the two articles has made me think about why.  So, here, in no particular order, are some reasons I'm negative about assessment.

1)  It's not actually about student learning, but rather it's always about accreditation, and especially about accreditation on a sort of business model, which sees us as turning out products (we call them "students" but assessment measures products) and asks how much value we've added as the products pass through our system.

When we're told, as I've been told lo these many times, that we have to "assess ourselves or someone else will 'do it' to us" with the "do it" being characterized in somewhat violent ways, that's coercive.  And it doesn't help me be a better teacher.  

2)  What's important can't necessarily be measured.  And what can be measured may not be all that important.  If you want to measure that students can learn how to think critically by taking courses that ask them to think in different ways, to learn about thinking, and to solve problems, how do you measure critical thinking?  You don't do it by telling students that you're teaching them critical thinking and then asking if they know you've been teaching them critical thinking.

I have a colleague who thinks I should write down all the goals for a semester long Shakespeare class so they can be measured.  I could begin, but the list would go on and on, everything from thinking about how racialized thinking in early modern culture contributes to our own racism, to how playhouse structure matters in theatrical experience, to ... well, I don't even want to go on.  How do I measure those things?  Some students will learn more, and be able to integrate what they've learned about race in US history, and something good will happen.  Other students in the very same class will deny that race can even matter because they think we're post-racial somehow.

3)  Administrators communicate that they think faculty have never ever in our lives thought about teaching, and can't even begin to think about teaching until we've been taught the official language of assessment-speak.  It may be that some few faculty never think reflectively about what the heck they're doing standing in front of 30 or 200 students, but I've never met a single faculty member who didn't talk as if they thought reflectively. 

4)  There's an ever-changing target, and it always involves a lot more work for faculty.  I recently wrote about spending hours in department meetings figuring out how we were going to apply for our courses to move from the old general education to the new general education system, even after we've all been told that the new system was specifically designed so that the top 70 courses from across the university would be the same top 70 courses in the new system.  That is, the system is designed to be almost exactly like the old system in everything but names.  We fill out the forms in good faith, spending hours to get the language just as the committee has told us it wants, and then the committee comes back and says that we have to do all those forms differently because what they said they wanted isn't it after all.

5)  There's an ever-changing target because the folks who claim to be assessment experts have little idea about what they're doing, and so keep constantly coming up with new ways of figuring out how to measure students as products because students as products aren't measurable.

I've been through assessment systems based on: portfolios, exit interviews, "embedded" assessment, and on and on. 

6)  It takes an administrator just a few seconds to demand hours of work from faculty and staff in any department, and none of that work seems at all helpful to our teaching.  Time is not flexible, not really.  If 20 people in my department spend an hour in a meeting working on assessment stuff, that's an hour we could be spending another way, and 20 hours altogether, which is half a work week for normal employees.


That's it.  I have to work on some stuff that actually may be of use to my colleagues.