Thursday, November 24, 2005

Advising - documenting, measuring, evaluating?

Recently, I asked a question of Dean Dad about how we measure or evaluating advising, and he was kind enough to take on my question here. (Thanks, Dean Dad!)

Faculty members in my department do a fair bit of advising; it's part of the job description and so has to be part of our evaluations. Each faculty member who's being evaluated in a given semester puts together a file documenting their work in various ways, copies of research, evidence of committee work, copies of teaching materials, and so forth. But consistently, we all seem rather lost when it comes to documenting our advising. As a result, when our personnel committee puts together evaluation letters, we tend to have only the most perfunctory information about advising.

So I'm looking for a way to document my own work better, for ways to mentor my junior colleagues, and for ways to evaluate files. And, of course, I don't want to add a bunch of work for anyone. The documentation and such shouldn't be painful for anyone, so it's got to be as natural and easy as possible.

So far, the discussion at Dean Dad's has been really helpful and interesting.

Dean Dad notes some basic ways we might monitor, including the basics of seeing colleagues in the office. Dean Dad, being a Dean and all, is worried about worst case scenario's, while I'm interested in a best case scenario: how can a colleague who's a fantastic advisor show that so evaluators know?

Here in my department at NWU, our lead advisor assigns advisees to each faculty member, which balances the work amongst those who advise in various areas. (The English Ed advisors tend to have more advisees, and a more complex program to deal with.)

Mathsophie suggests that using questionaires and advising evaluations would be useful. I think that's a great suggestion. We already do exit interviews with our majors as part of our ongoing departmental assessment, and could perhaps ask them about advising in the process. Or perhaps we could ask them to fill out a short survey. (The exit interviews are sometimes done by non-tenured faculty members, so they aren't in a legal position to evaluate other non-tenured faculty members, but a survey could be seen only by the advisor and personnel committee members.) Thanks, Mathsophie!

BitchPhD adds that advisors could forward/copy emails to people responsible for evaluations. Since I have a feeling we all do a fair bit of email advising (I do), we could at least make some hard copies and put them in a file to document how we handle typical problems. That seems like a great idea, which wouldn't add that much to anyone's work (people are already putting together files, and a few extra pieces of paper isn't too much to print out, nor too much extra to read over).

She mentions the idea of asking advisees to write letters of support. I know I did that for several professors when I was a graduate student. I wonder how well that would work at the undergraduate level? Would students feel coerced? Should the personnell committee solicit letters from students suggested by the faculty member?

She also suggests that we could look at students' success at graduating and fullfilling requirements. That, too, sounds like a possibility; I wonder if it could somehow be combined with an exit survey? Because the student would be graduating, it seems less likely to feel coerced or problematic for the students, doesn't it?

Thanks for the great ideas, Bitch! (Gosh, that feels sort of weirdly rude to type! My Mom would wash my hands off with soap or something.)

So far, I'm getting some really good ideas! My thanks to Dean Dad for starting the discussion!


  1. Anonymous6:45 AM

    Just FWIW, at my last job we asked advisees to fill out surveys on their advising experiences, and return rates were abysmal. There may be some better way to do it than what we did, of course, but it didn't generate anything remotely meaningful that we could use. But I think soliciting letters from undergrads would work perfectly well (I've only worked at places with all undergrads, and the students seem to take the responsibility quite seriously).

    It is a difficult thing to measure. At my last job, we constantly faced the issues you describe here. One of the things that was relevant was word of mouth - the central advising people got to hear who was most requested as an advisor, and who was most NOT desired as an advisor. But that's pretty ad hoc and hard to quantify/prove.

    The suggestions here sound very good, and I'm going to have to check out the discussion at Dean Dad's!

  2. Thanks for the suggestion, New Kid!

    I think if we have students do a survey while they're in the exit interview mode (just before, just after), then we'll get a reasonable response rate. (I'm a little leary of student evaluations overall, since I've seen a fair number of sexist comments from students about female professor's physical attributes on teaching evaluations, and have also read some research that suggests they aren't really reliable measures. Still, they might be worth looking at.)

    I think our lead advisor tries hard to balance advising assignments, although he does let students with a strong preference choose. I don't know that most new majors have enough grapevine contact to have strong preferences, though.

    And yes, by all means, drop by Dean Dad's place. He's really interesting.

  3. Hiya

    I posted over at DD's too. I may be overly sensitive to such things, but I feel a bit concerned at forwarding email conversations between and student and an advisor to a third party.

    I know you're interested in best-case scenarios, but setting that aside would you have any concerns regarding privacy?

    As I wrote there, I'm pretty sure all emails to professors are basically property of the university (the employer) anyway. But I still would want to warn students that every advising email will be shown to a third party.

    I guess you could anonymize the emails - take off names and headers and suchlike.

    or ask permission of the student before forwarding particular correspondence.

    Institutionalizing the practice of forwarding the emails, or a selection of emails, seems rather risky - that seems like it could be a lot of information that should be stored securely somewhere, with access limited to specific people. As a database it would have to be administered in some safe way.

  4. You're welcome! I'm glad my suggestions seemed helpful.

    Is it wrong that I'm always amused by the--completely unintended--effect that people feel weird addressing me by my nom de plume?

  5. Good point, Ripley!

    I guess when I'm thinking of forwarding emails, I'm thinking of copies of emails I send out to my students, rather than emails they send to me. For example, I usually send my advisees a monthly heads up sort of email about things they should be doing/thinking about or whatever.

    But you're absolutely correct that anything to or about a specific student would have to be submitted in a way to guard confidentiality.

    Since the evaluation files for faculty are supposedly confidential, and only seen by our personnel committee, chair, and so forth, the materials inside would also be supposedly confidential.

    Thanks again for the responses, folks. I appreciate the help.

    PS, no, BitchPhD! I live to amuse!

  6. Anonymous6:53 PM

    I am glad I am not the only one who finds using student letters to be a thorny issue. In fact I consider it to be unethical. For example, should one ask a student who is currently in a class? How about a student who is not currently in a class but might join a class next semester? How about only ask students who have graduated, but might require a letter of reccomendation later?

    I find the idea about as healthy as asking when it is OK to date a student: I don't think it is ever OK to ask a student for a letter of support.
    El Campesino

  7. Anonymous9:40 PM

    Hmmm. As someone who's worked at small teaching colleges, soliciting letters from students is extremely common (at least when it comes to going up for tenure, or even third-year reviews). At my current institution, if I remember correctly (will have to figure this out soon!), the department chooses X number of students to ask for a letter out of all students who've taken your classes. That will include some people who are graduated and won't be taking more classes with you again, and some who will probably take more classes from you. But it's all organized from the department's end - they generate the list and ask the students, and get the letters back from them (whoever feels like responding - and there's no guarantee that the letter will be in support). I'm not sure that I, as the reviewee, would even ever see those letters (though the reviewee did see them at my previous job). I don't see any coercion of the student going on, since the reviewee isn't asking students directly, and will never know if a student was asked and declined to write a letter. To play devil's advocate, I see it as a great opportunity for students to be involved in the process of evaluating professors and playing a part in the tenure process - which, at teaching schools, is a pretty important thing. Sure, there are problems with all student evaluations. But I've always seen student letters as a good thing, not something that puts the student in a difficult position. (B/c, as I said, the reviewee themselves never asks students directly.)

  8. Anonymous2:47 PM

    Some sort of award ("Advisor of the year") might help identify especially good advisers.