Tuesday, September 15, 2009

Are Interviews Worth It?

At Leiter, Richard Holton of MIT wonders if the recent backlash against interviews isn't overblown. (This issue was discussed at PEA Soup and the Old Job Market Blog, but I couldn't find where.) He cites someone who researches interviews, Allen Huffcutt, who told him:
Some may argue that academic positions are unique so the results of research do not apply to them. I would argue against that. The process of the interview is to identify key characteristics needed for a position, and then ask questions to assess those characteristics. The research out there has been applied to a vast range of job types, a number of which have very similar key characteristics to faculty positions.
I guess I would agree with this. I suspect that there is nothing especially special about philosophy interviews that would make them immune to the larger body of research surrounding interviews. (I don't know who's been saying there is.) I also think it's an absolutely terrific idea to consult the research in a deep way, correspond with people who study this sort of thing, and try to get to the bottom of things. Huffcutt goes on:
What I might suggest is a carefully thought out approach to hiring philosophy faculty. In the job talk, looks for key elements related to presentation. E.g., are they well organized? Do they find ways to make the material interesting and relevant to the students? Do they deliberately bring people into the presentation (e.g., ask them questions) or just stand there and lecture the whole time? Do they handle questions well?
Interesting thing about several of these key elements: they are not normally elements of a philosophy job talk. (I distinguish the job talk from the teaching demonstration.) I've seen some successful job talks that would have been incomprehensible to students. While I don't recommend deliberately disengaging from your audience, it would be weird to pause in the middle of a job talk to ask them a question.

And suppose the candidate was disengaged, over the heads of the undergraduates, and handled the questions poorly during her job talk. What would that prove? Maybe she had a bad day. Maybe she doesn't handle high-pressure, whole-future-is-riding-on-the-outcome interview situations that well. None of that is evidence that she's a bad philosopher or that she'd make a bad colleague. None of it would outweigh the evidence in her dossier that she would be a good philosopher.

Gilbert Harman makes a point in the PEA Soup thread that bears repeating. Suppose you are told that your candidate has really effed up her interview at some other school but did okay at yours. It wouldn't affect your opinion of her--you'd ignore it. But if she effed up yours but did okay at some other school, you'd never hire her. This makes no sense, and illustrates why these sorts of interviews have been called vivid noise. They introduce unreliable, distracting information that seems to be neither unreliable nor distracting.

More Huffcutt:
During the interview, look for key elements pertaining to social interactions. For instance, using the behavior description format, you could ask them to describe a time when they had to handle a difficult student, a time when they went out of their way to help a student, etc. Same for their interaction with other faculty.
This is maybe a little better. I think it's worth it to try to find out what kinds of questions will elicit useful information. But I still don't see how this addresses the Harman point. Suppose the person fumbles the "difficult student" question, but you've got a bunch of evidence from her dossier that she's a dedicated and talented teacher. What does that tell you? How would you react if you learned that she fumbled somebody else's "difficult student" question but aced yours? How would you react if she aced yours but fumbled somebody else's?

Holton ties it together:
discussing someone's paper, which is what many APA interviews and on-campus talks consist in, is rather like a mix of a behavioral descriptive interview and a situational interview; you spend much of it finding out how the candidate responded to various philosophical problems (BDI), and then you find out how they would respond if someone said something like what you then say (SI).
While I appreciate the effort and admire the empirical approach to what is clearly an empirical question, I guess I just don't see why I should think that this sort of thing is likely to yield useful knowledge. What is the useful Behavioral Descriptive Information supposed to be? And how is it useful to know how the candidate would respond if someone said something like what you then say? And what distinguishes the BDI from the SI in this situation? Maybe I'm dense, but I just don't see how that would result in helpful information about the candidate that would be more reliable that what's already in the her dossier.

--Mr. Zero

21 comments:

Anonymous said...

The interview serves a number of useful functions. First, it has psychological value for the hiring department. After such a cumbersome and draining process, the hiring department feels comfortable constructing the myth that they've hired the "best candidate". Of course, this is almost always false, but they'll feel warranted in believing it. The myth is important. It helps the members of the hiring department to respect and help their new colleague. It's rather amazing how quickly the myth is constructed and how quickly the past is forgotten. At my university, our last hire was quite obviously the *worst* of the candidates invited to campus, but people have long forgotten.

Second, interviews give the hiring department a feel for the candidate's basic social graces and instincts. A candidate's Neanderthal social attitudes (rather common in philosophy) might pop out and disqualify him or her at the interview. If the candidate and committee really hit it off, they might feel as though they have a future conversation partner.

Third, interviews allow departments to help ensure ideological purity or centrality in focus. Some departments advertise themselves as doing philosophy *this way* and not *that way*. It isn't always obvious from the CV that the candidate fits the department. The interview gives a better picture.

Xenophon said...

Several comments, which in sum don't support one approach or the other. But I thought your original post was interesting and thought-provoking, so I'm responding to your points.

First, interviews for research-oriented schools are different than those for teaching schools. Teaching schools would be crazy to use the Princeton method of hiring based on dossier alone. For research schools, this might be the best approach.

Second, I did a job talk last year at a teaching school to an audience of mostly students (though all the philosophy faculty and a couple of others were there also). My research is in a field that no one there had any background in, and I used my standard research talk, although with more explanation of background and otherwise slimmed down to fit their schedule. About halfway through the lecture portion, I could see a sea of fische augen, so I stopped and asked if people were following, and if they had questions. A little discussion and everyone looked much happier. At the end I got good questions from students that demonstrated they really did understand. So I'm inclined to say that it is important to see that a candidate can demonstrate more than the ability to lecture, that they can engage an audience.

A related point: at teaching schools, the research talk is really a second teaching demo. The committee can get an idea of your research from the writing sample: the research talk gives them further information how you interact with students.

As for Gil Harman's argument: I think it's perfectly warranted to put greater weight on evidence from experience than evidence from hearsay in such a case. Sure, my colleague at school X says this candidate flubbed the interview there, but I don't know all the factors that affected that performance. I do know how we're running the search at my school, and I think it's well-designed and well-run, so I weight the performance at my school more heavily than the performance of the same candidate elsewhere.

Behavioral interviewing is one of the dumbest ideas than HR departments ever came up with, so I think Huffcutt's off base here. "Describe a time that a student complained about you to your chair, and how you acted to it." "Well, gee, that's never happened." "Oh, you don't have an answer to that one, too bad." That kind of thing comes up all the time. Or how about "Describe a situation where you encountered a problem, and tell us how you dealth with it?" Could you be any more vague? Some behavioral questions are alright, but most suck so it's better to avoid them as much as possible, I'd say. (I actually got the second one from an HR drone, for a university job.)

Anonymous said...

Off topic, but I thought I'd ask here because people here seem to have opinions.

In spent a semester doing double duty, I was an adjunct but also had a full time position on the far side of town. I can imagine a search committee having one of two reactions.

Reaction 1: Dude, so and so taught six courses and still managed to get research done. Rudy! Rudy! Rudy!

Reaction 2: Ewwwwww, adjunct. That's gross.

Should I list the adjunct position on the CV or should I keep it hidden? Or, should I try to guess who is too snobby to hire someone who once worked as an adjunct and try some mixed approach.

Also, will referees get their fucking work done already. I managed to referee three papers over the summer (all within two weeks, I might add, including two that I gave positive responses to in spite of the fact that they probably make some of my work unpublishable since they contain similar enough arguments that my work won't contribute to the literature). Following Andrew Cullison's lead, all were done in two weeks. Meanwhile, I've received news on none of the seven papers I have out under review and they've all been out under review since well before the summer. WTF!

Dr. Killjoy said...

Think the pedigree shit is bad? How about having your fucking face be a deal breaker? Just one more worry for all you uggos out there.

“Facial Discrimination: Extending Handicap Law to Employment Discrimination on the Basis of Physical Appearance,” Harvard Law Review vol. 100 (1987).

Anonymous said...

Xenophon -

I think you're missing what behavioral interview questions are testing: the ability to generate plausible bullshit answers. This is certainly what "what is your biggest flaw" sorts of questions are ultimately testing for. Modern workplaces of all kinds require this skill in spades, so it makes sense to weed out people who don't have it.

Anonymous said...

C'mon, people. The purpose of an interview is to try to figure out "fit," or in academic terms "collegiality." In other words, can you stand to be in the same department with this person for an hour/semester/forever? You can argue whether or not the interview accomplishes that (depending on length of time spent with the candidate, the number and types of interactions the candidate has with faculty, staff, students, etc.), but all the materials required by the search committees are designed to determine if the person's scholarship and background are appropriate. If you're bringing in people to campus who can't do the work on paper, then you've got bigger problems than the interview mechanism.

Anonymous said...

To my mind the central problem is the fact that I have two competing attitudes, neither of which I can give up.

(1) Interviews are not a reliable method for ranking candidates' quality (where quality might be thought of in terms of what best fits with the needs of a particular institution).

(2) There is no way in hell I'm going to hire a colleague in my small department without at least first meeting them in the hopes of weeding out the douchebags.

Dr. Killjoy said...

In all seriousness, even if we were to suppose the noise argument to be bunk, there still would be little to no good reason to prefer in the initial hiring phase face-to-face interviews over phone/conference call interviews. The latter should be just as effective as the former and has the distinct advantage of being fucking free for everyone involved.

Grad students wouldn't have to shell out hundreds on plane tickets and hotel rooms, and hiring departments could use the money otherwise spent on funding the hiring committee's APA travel to flyout half a dozen of the candidates selected from the phone interviews. This would all things considered cost the department no more than normal but would better promote the goals of the hiring process, i.e., greater number of viable candidates with whom the committee can interact in the most productive and informative environment (job talks, teaching demos, etc.).

Given that in most cases there will be little movement in rank from initial selection for interviews to selection for flyouts, why should we continue with a process that does SFA more than unduly financially burden those already financially hard up?

Just think, if hiring committees were simply to pick up the fucking phone, the Eastern APA might be able to become an actual philosophy conference rather than the nightmare sad-sack cluster fuck it is now.

Anonymous said...

Anon 1:59,

If interviews are a way to weed out douchebugs, how come there are so many of them in philosophy departments?

Asstro said...

Anon 10:06:

Yes, list your adjunct work, unless it's at a college of absolutely no or very little standing. If it was just a way to earn a buck, scrap it. If it was a place that you think is of reasonable prestige, consider even getting one of the TT people to write you a letter.

Also: don't sweat the referees yet. It's bullshit, yes; but journal editors will probably be sensitive to your concern and pressure their referees with papers by prospective job candidates to get back to them faster. I know I would. Moreover, you'll probably hear soon from a few of those places... and as the referee reports roll in, be sure to notify the schools that have your application if you get a publication.

And finally, fwiw: I would never, in a million years, hire a candidate sight unseen. No way, no how.

Polacrilex said...

Dr. Killjoy's comment has led me to ask something I have wondered since I interviewed at the APA last year: does anyone know of any candidate who did a phone interview instead of an APA interview (in a situation where one could do one or the other) and ended up getting the job? I realize that an on-campus interview most likely followed the phone interview, but I'm most interested in that initial interview.

The Eastern APA has become a waste of time, money, and emotion. If a school wants a candidate, i.e. un- or under-employed philosopher, to come out to an overpriced hotel to go through that illogical mess, then the school should foot the bill just like they do for an on-campus interview. Spare us the traditional gauntlets.

Anonymous said...

Anon 4:17 writes "If interviews are a way to weed out douchebugs, how come there are so many of them in philosophy departments?"

It doesn't weed all of them out obviously. I merely said it is my hope to weed them out. Avoiding just one might the difference maker for my quality of life.

I like what Asstro said: "I would never, in a million years, hire a candidate sight unseen. No way, no how."

Anonymous said...

Polacrilex,

In case this information fits what you are looking for, I got my job last year with neither an APA interview nor a phone interview. The first contact I had with the school was when they called me for a campus visit. I was one of three candidates (they were definitely not going to fly more out). So they saved money and time, but I feel they took a gamble. (Hopefully they feel like their gamble paid off!)

Anonymous said...

I had a phone interview instead of an APA interview last year for one particular TT job. Then came the campus visit (4 candidates), and then the job offer. So the process can work without the APA interview.

IMO, phone interviews accomplish just about all one needs to accomplish in an interview (minus all that ungodly cost to go to the APA!).

Anonymous said...

My job came from a phone interview as well and my sense was that a phone interview + campus visit should suffice. If you really want to sneak a peak at a candidate before hiring him/her, have a skype interview. In my opinion, APA interviews are a serious waste. I had one at the last Eastern and I can't imagine that they learned anything they needed to know or thought that they needed to know by meeting with me face to face in their suite that they couldn't have learned by asking me over the phone. For example, they knew that I was already very low down on their list. They knew that nothing short of a miracle was going to move me into the group that was going to get campus visits. (Luckily, I had a paper to give so it wasn't a total waste.)

Xenophon said...

Polacrilex,

I've had a number of phone interviews over the years. I'll say this, it takes a very different skill set to do well on them than it does in-person at the APA.

It's hard to keep track of who's talking unless they're nice enough to tell you every time they talk.

It's hard to know when they're done talking, and the candidate sounds like an ass if he starts talking right when someone else is about to, over and over (in person this doesn't happen as much because people give verbal clues when they start talking, like opening their mouth or leaning forward).

Most departments use speakerphone, so there's always one member of the committee who you can never really hear.

My suggestion is that every search committee member sit in their own office and do a conference call. That way everyone's at the same disadvantage and you'll understand better the challenges that the candidate faces when he's the only one not in the room.

I've developed a number of tricks for phone interviews. I put on a coat and tie. They can't see whether I'm sitting at my desk in my underwear for the interview, but I feel different when I'm dressed nice. If I can learn who I'll be talking to ahead of time (always good to ask when you set the appointment), I make a seating chart with their name, AOS, PhD-granting institution, and photo cut and pasted from the school's website. That way when someone's talking I can look them in the face.

Finally, a question. Does anyone know an easy, cheap way to do videoconferencing over the internet? That would be much better than a phone interview, and should be cheap. Candidates would rather shell out 20 bucks for a camera to attach to their laptop than have to go the APA. One interview a couple of years ago tried to do this, but they wanted a fancy method that required every candidate to talk to their college's IT dept to set up, and they didn't realize that people who already had jobs couldn't really go to IT and ask them to spend money so the poor philosophy VAP could get a job somewhere else. That ended up not working, and the dept reverted to a phone interview in the end.

Anonymous said...

To add to the data:

At the last minute, I was unable to make it to an APA interview, so I ended up having a phone interview with that university instead. (They then flew me out - I wonder how often it happens that some candidates have phone interviews and others personal ones, and what difference it makes? That could be a source of some interesting data.)

I second what Xenophon said about the difficulties with phone interviews, though in principle it seems like we could find ways to overcome them.

For what it's worth, my current department just does campus visits (3-6 candidates), and I can't really see what interviewing at the APA would add. It would probably make us look at additional candidates who were iffy on paper, but then it's easier to see hiring those candidates as allowing sparkling personal interactions to override hard facts, rather than finding the hidden gem.

Also, I suspect the APA talks would be better attended if we got rid of all the hiring stuff.

Anonymous said...

Since this is the blog in which "issues concerning the profession of philosophy are bitched about" and since we're a talking about interviews, I would like to bitch about a related, but somewhat tangential matter:

WTF is up with the APA not reserving rooms at the group rate for the 26th of December? The meeting is from the 27th to the 30th. Those lucky enough to have interviews on the 27th should probably arrive on the night of the 26th. Of course, there is no special rate (student or otherwise) for that night. If a candidate wants to avoid traveling to NY the morning of their interview, they will have to pay an extra $300+ dollars for the privledge.

Seriously: WTF was the APA thinking? Can we have a post about this? Who do I complain to?

Anonymous said...

something that does not seem to have come up in any of these discussions (at least not that i've noticed) is this: whether or not an interview/job talk/campus visit is useful is a separate matter from whether the department takes this information into account when making a decision. in the hiring processes i've witnessed in my program, there has been no correlation between, for instance, outstanding job talks and who gets hired. we have hired socially awkward candidates who gave terrible talks over cheerful and communicative candidates who gave outstanding talks, purely on the basis of how they look on paper. if you're going to ignore the information that in interview can provide, then you really shouldn't bother with them.

Polacrilex said...

Thanks to everyone who answered my question about phone interviews over APA interviews. Although I was already convinced, I am further convinced that attending the E-APA is a waste of time, money and emotion. I realize that this will not happen, in part because candidates will see it as an advantage to attend over those of us who do not, but I wish all job candidates would simply boycott the E-APA (as far as the hiring process goes; the conference itself seems completely separate). I remember being asked last year by some schools I had applied to: "Are you planning on attending the APA? If so, we would like to interview you there." What if I would have said no? If they would grant me a phone interview anyway, why waste the time and money?

PFS said...

Policrilex,

There is no option about phone interviews or APA. If they ask you if they are going to APA, they are just trying to be polite; they are not dangling a secondary invitation for a phone interview. Perhaps, you could say that you would be willing to go but that it would be easier to do a phone interview, if done judiciously.

BTW, I had a situation where I had 1) a phone interview->on-campus interview->TT job offer (3/3 teaching load); and 2) an on-campus interview (without a preliminary interview, phone or APA)->TT job offer (3/3). I've also had a 3) VAP job offer (3/3) (without any interview whatsoever). So, there really is no standard procedure. To show that I am not a special case, I have to also say that I come from a non-Leiterrific graduate program.