When I got my first TT job, the SLAC I worked at was just starting into an accreditation cycle, and everything was all about assessment. Basically, the accrediting organization
The idea of a portfolio is that students would collect evidence of their work together in a folder, and towards the end of their college career, they'd share this with the college, which would look it over. From the student folders, the college would be able to learn if the college was actually teaching the students what the college said it was teaching students. One of the keys to such assessment is that it's NOT about evaluating student work or learning, it's about evaluating the school. That is, the system imagines that the college does things to "add value to" students, and that portfolios are a way of measuring the "value added." In the system my SLAC adopted students can't be graded on their portfolios, and that portfolio quality can have no impact on graduation or grades; as I understand it, that's typical of most portfolio assessment systems. So students have no stake in the portfolio system. If they do a half-way job, no problem for them. If they hand in an empty folder, there's no problem for them. But for the College, it matters.
I've used "the college" here, but in reality, the college as a corporate entity isn't what does the teaching, and certainly isn't what does the evaluation of the portfolios. So, what happened was that each faculty member was required to help students (primarily advisees) set up a folder, and
I was a new, untenured faculty member who'd never really had any liberal arts experience, so I nodded with whatever enthusiasm I could muster after teaching my four classes that term, and tried to do what needed to be done. I encouraged my advisees to put materials in their portfolio; any materials they thought would show what they'd learned could go in, so my advisees put videos of class presentations, recordings of music sessions, programs from performances, papers, art work, papers, you name it. We were told that our students could use these portfolios to get jobs! To impress people at cocktail parties! Yes, we were told, these portfolios should really be of great benefit to students!
Then graduation time came. The administration told us that we needed to evaluate the portfolios of students graduating in our department, and write up a report about what we were doing well or not. I didn't see any other reports, but my department was doing brilliantly! We were stunning! Our students were miracles of learning and erudition!
Fast forward several years. Take a geographic jump to a colder clime.
When I first came to NWU, the school was also in the throes of accreditation, and by golly, the accrediting organization also strongly suggested we adopt a portfolio system. Being a much larger school, NWU eventually adopted an electronic portfolio system, into which our students might enter anything they wished so long as it's in Microsoft Word (dot doc) format; no need to worry about artwork or performances! And instead of individual advisors or departments reading these portfolios, NWU actually named an Assessment Guru to put together a team every year to read and evaluate.
At the tiny SLAC, an advisor could do a lot to encourage students to actually keep a portfolio; at NWU, there's no incentive for students to keep a portfolio. It adds extra work to their lives without any benefit. Even the pretend benefit of keeping materials together for potential job interviews fades quickly when, within a few short weeks of graduation, students lose all access to the electronic portfolio they've put together, and within a short time thereafter, their work is purged altogether.
Being non-tenured, I nodded with hopeful enthusiasm, and did my best to encourage my advisees to put documents into their portfolio. The Guru encouraged us to give assignments in our first year writing AND other classes that would fit in the portfolio slots, and to require students to put these assignments in their portfolio. And being non-tenured, I complied. The Guru asked us each to let one of the student workers come spend half an hour doing a portfolio presentation for our students. Being non-tenured, I complied. (The presentations were not universally well-done, alas.)
I've toed the party line. I've told students that they should do the portfolio for the university, and also try to benefit from it themselves, by using it to reflect on their university education. I've tried to meet student cynicism with good cheer, with explanations about how reflecting on their experience would be valuable to them. I've given assignments appropriate to the portfolio, and required students to turn in print-outs showing that they've entered documents into the electronic system. Mostly, it's just extra effort to think of ways to get students to put things in their portfolio, make a note of some sort when they do, figure out how that portfolio bit fits into the class.
Doing all of these things gets me no credit; there's no place on MY departmental evaluations that cares about my attempts to work with the portfolio system in various classes. Being known to not do these things could, I suppose, get me some frowns. Making large noises about not doing them would earn me some mutterings.
In the past several months, the Guru has said (publicly) that the portfolio project really isn't working. It seems perhaps (I've only heard hints) the portfolio analyses show that we aren't actually doing something we say we're doing. And the students aren't taking it seriously in high enough numbers. And maybe we're just going to dump the system.
Ten plus years down the line, people (at other schools) I talk to tell me they're dumping their portfolio assessment system because it's unmanageable, doesn't work in some way, adds costs, etc.
So what I'd like to ask all the assessment folks out there: Wouldn't it have been a good idea to actually figure out if portfolio systems really WORK and are do-able before forcing all of us to engage in this process?
Now I'm beginning to hear rumblings about other modes of assessment. These, I'm hearing, will be super effective! They'll be meaningful to our students! We'll know what the "value added to" each student is!
Can I be forgiven for feeling just the slightest tad cynical?
Can I be forgiven when the Marxist inside me wants to know why we're unquestioningly treating human beings as products?
Can I be forgiven when I want to ask what the student is contributing to his/her education?
Because I really do want to profess well, I will try to be reasonably open to the next coming thing.
Because today I feel tired, I just want to smack my head against the wall for comfort.
And I want to ask of the Guru, what research have you done to demonstrate that THIS mode will work more effectively than the old one?
While it likely wasn't your intention, I read this post as being so hopeful! Not hopeful of administrators/assessors, but hopeful that professors (you) care so deeply about your students, how well they're learning, and how well the college is responsive to their needs.
ReplyDeleteImagine -- all these pseudonymous blogs where we are all sharing our deep dark secrets (like, we care about each other and our jobs, we care about our students and teaching, we care about the direction of education, we care about how our lives look) -- imagine what it would be like if we acted in our everyday lives like this is what all our colleagues would write on a blog. Maybe we could actually get together and do something.
Thanks for writing such a great post.
In my last position in a recently refurbished writing program, I was on the program assessment committee, and soon got very discouraged about what could even BE assessed about writing or even humanities more generally...that which we do--teach subtlety, nuance, complexity, is not easily quantified, especially when it so hard to even describe in a clasroom without modeling.
ReplyDeleteWhat we found was that we could get summative data that wasn't formative for the teachers who might want to use the data to help their teaching, local measurable data that didn't really measure our actual learning objectives, or global measurable data that, in order to get positive results, would actually force us to teach in ways we knew to be counter-productive to our own mission.
The one assessment exercise we ran, and which gave us rich, complex results, came back from the university Guru with a frown and the pronouncement that the data could be used against us, and that this was merely bad assessment. Forget te fact that it helped crystallize our own learning goals and form more coherent evaluation strategies and the pedagogies that would work toward them.
What that experience told me was that the humanities and writing still resist quantification, and therefore commodification, which is both its greatest strength and may, in this late-capitalist, neo-liberal, corporate-university phase, be its doom. It's why I am committed to thinking about ways that we can use the humanities to challenge that phase, since not doing so may in fact be complicity in the demise of our own epistemological ways of being.
I'm surprised to hear that access to portfolios is gone after graduation... in my state the system is a state-wide one that is accessible to any resident... at least that is what they told me...
ReplyDeleteThere are significant problems with assessment at the college level. The real issue is that there is no solid research done in terms of assessment methodology, nor is there a good way to measure end results that takes into account the level of the student when they arrive at the college.
Frankly, I'm not at all sad that I'm done with my 3 year stint on the assessment committee at my CC.
Bardiac, you've done everything possible to make this system useful, but what is the university thinking by wiping out the portfolios and thus negating any long-term benefit for students?
ReplyDeleteIMHO, the people who propose the Next Revolution in Assessment (and make the big bucks as consultants or administrators) measure success by how many departments/schools/instructors they can get to carry out their program; their success is measured by a program's wide adoption, not by its actual effectiveness at doing what they claim it will do.
I'll be interested to hear if the Guru has an actual answer to the question about research and proven effectiveness.