Thursday, May 15, 2014

Review round-up: The good, the bad, and being helpful.

I've had some mixed experiences with peer review this month. The bad: a journal that held my paper for three months, then returned it saying it was "inappropriate" for the journal with absolutely no reviewer comments. Which is bullshit. It's not "inappropriate" (at least not as I understand that word) for the journal, as they happen to have a special issue coming out on the very same (general) topic. (There was no public CFP for that special issue: the papers included are really, really, really obviously invited, all bigwigs who write about it all the time.) Methinks inappropriate in this case means, "you were not invited to write a paper for our special issue, and we don't need your stinkin' anonymous nobody paper." Which, you know, they could have told me that in a lot less than three months, so I could have moved on with my stinkin' paper.

Which reminds me of a paper that got outright rejected seven times in seven days -- I appreciate that the journals were clearly not interested and said so in a timely manner. (It was finally accepted.)

The good: I got a paper back last week with some of the most helpful, most detailed comments and suggestions for revisions I've ever received, which I am quite sure will really make the paper better. In re-reading my paper, I can clearly see what the reviewers meant, and how their suggestions can be implemented. Plus, the reviewers understood what I was doing, and their suggestions were in the spirit of making that more effective. My sole complaint there is the journal's use of AMA style (numbered references), which is a total pain when you're making revisions that will require reordering and renumbering all the references. (Maybe there's a way to make that happen automatically, and maybe I need to finally learn how to use Scrivener, but I haven't yet had time to do that. Kinda busy trying to write papers.)

So, it made me think about my own reviews for papers, and how much time I spend on them, and how well I write them. I guess one issue for me is that I don't have a lot of time, but I do agree to review papers a few times a year because it is part of the process, and a contribution to the profession.  But maybe I haven't thought enough about how much my reviews might assist authors (rather than journals), especially authors like me, who are junior and having to crank out a lot of work for tenure, working in virtual isolation from peers, and without adequate support systems in place to get useful feedback on papers. (I mean, I have friends in philosophy, but there's only so much I want to impose on them. My colleagues would not be much help.) I've reviewed some really terrible papers, and some really pretty good ones, and I think really hard about my judgments on their publication-worthiness. But now I'm really thinking my reviews could use some improvement, and I should focus more than I have been on being helpful to the authors.

~zombie

32 comments:

Anonymous said...

Get EndNote. Automatically format all your references in whatever style you want, re-number, re-order, whatever you need. Extremely easy to convert a paper from one citation style to another if submitting to a new journal. Should take an experienced user of computers about a day to learn how to use.

Anonymous said...

From my (limited) experience, journals are much more likely now to ask referees to report back within 1-2 months. I think this overall is a good thing, but perhaps it will also diminish the quality of the received reports? I'd suggest a law inversely connecting the time to review and quality of the comments, call it Clark's Law, but I've seen and heard plenty of counterexamples.

Anonymous said...

Or use BibTex.

Re: Reviewing. Here's my policy: agree to review within a month or don't agree. (I do ~6/year) Always provide detailed comments under the assumption that the editors won't take your word for what's bad (or good) about the paper, despite the fact that, of course, they almost always do. I treat it like I'm writing up comments for a friend.

Anonymous said...

just curious, was the good review from Phil Quarterly by chance?

zombie said...

Not Phil Quarterly. It was a specialist journal.

My school licenses EndNote, so I've downloaded it and will try it!

Anonymous said...

"I guess one issue for me is that I don't have a lot of time, but I do agree to review papers a few times a year because it is part of the process, and a contribution to the profession."

Nobody really has a lot of time to write reviews, and that's one reason they often take so long.

I used to wonder why my advisor, for instance, would complain about the time it takes, considering that he only taught 2 courses/ term and seemed (to me) to have tons of time. But reviewing papers was always less important than his teaching, his research, the graduate students he was advising, and his personal life.

I assume the same is true for everyone else.

Anonymous said...

6.20: you take up to a f*king month to respond to requests for review? Stop it. Any editor who hasn't written you off after 2 weeks shouldn't be editing. You should be responding within 48 hours.

6.56. Most of us assign a pretty low priority to mowing the lawn. That's not a reason to let it drag on. That's a reason to get it out of the way.

Anonymous said...

6:17 - I'm pretty sure 6:20 is referring to a deadline for having a review completed, not the amount of time taken to agree to review in the first place. So s/he only agrees to review if the review can be completed within a month, and otherwise does not agree to review. This seems eminently reasonable to me.

The UnDude said...

6:17,

You're the reason that we can't have nice things. 6:20 was saying "agree to review if I can complete the review in a month" not "take a month to respond to all queries regarding reviews"

Anonymous said...

"6.20: you take up to a f*king month to respond to requests for review? Stop it. Any editor who hasn't written you off after 2 weeks shouldn't be editing. You should be responding within 48 hours."

First off, I think you're misunderstanding 6:20.

But more importantly, please keep in mind that your article is not nearly as important to other people as it is to you. Take a breath, count to 10, calm down, and realize that even if it did take someone a month to agree to review a paper, that's really not that long. If you submit the paper at the start of a semester, and it takes someone a month to agree to review it, it's still entirely possible to get a decision back by the end of the semester.

Anonymous said...

This is 6:20.

6:17: as others noted, you not only flew off the handle, you also misunderstood me. I agree (or not) within 24hrs, and I agree to review the piece within a month, i.e., to have the review completed within a month. I make sure to say this in my response to the request, so that I've set myself a firm deadline. Calm down.

Anonymous said...

Has anyone heard from Bowdoin?

Anonymous said...

They've hired someone who is currently an asst prof at Stanford, polisci dept

http://www.bowdoin.edu/digests/entry.jsp?timestamp=20140224083525&dow=2&type=student&week=201409&author=jsellers

Anonymous said...

Cambridge has posted a 1-year temporary position in ancient philosophy (Classics faculty); it's not on philjobs yet and not sure if they will post it there. http://www.classics.cam.ac.uk/directory/vacancies

Anonymous said...

Thanks for the link, 6:12am. But I had in mind the Bowdoin post-doc, which was advertised after the posting of the link you gave.

Anonymous said...

General question on topic --- what has been your experience in terms of response times from the editor upon immediate submission of an article?

For example, I submitted an article to a journal about 2 weeks ago and still haven't heard from the editor even in terms of a follow-up "we've got your article and will get back to you" email.

Anonymous said...

5:56:

Unless there is some online mechanism for checking on your submission, I would send a polite email to the editor asking about the status of your paper.

zombie said...

5:56 -- I never get something like that. At most, I get an automated response right away. But I guess all of the articles I submist are online, thru ScholarOne or some such. I check online for the status.

Anonymous said...

5:03 AM / 2:40 PM

This particular journal uses email correspondence only. I suspect the editor is busy -- but will follow your advice and send a polite email query.

Patience is a virtue, but so too timely responses...

Anonymous said...

7:28,

A good rule of thumb is to never send out articles at the end of the semester.

Anonymous said...

Start using Scrivener! The learning curve is not steep at all. Wouldn't help with the referencing problem, I don't think, but do it anyway and you won't look back.

Anonymous said...

Ok, so I posted here about a week ago (May 30, 5:56) regarding a paper submitted two weeks ago (at the time) without a response. Another week has passed and still not a peep--even though I sent a short email query asking the editor if s/he had received my paper.

I recognize what everyone is thinking - people are busy, etc., however, in my opinion, this is patently unprofessional and to be honest, my only interest at this point is retracting the submission and moving to at least a more organized journal.

Any thoughts?

Anonymous said...

1:05:

Unless you really, really need to get stuff accepted very quickly, I'd wait. I can think of multiple times when I submitted to journals (not run via some online thing that sends an automated message once you submit) and it took the journal multiple weeks to acknowledge receipt. In one case, no such receipt was ever sent (but the paper was eventually accepted for publication).

I conjecture that this is especially prevalent with journals without online account that have graduate students as managing editors. Some of these journals are even very good (PPQ, Archiv).

If I were you I'd wait.

Anonymous said...

You don't need a response from them to retract your paper. So if that's the decision you've settled on, just tell them. You can send it elsewhere without having to wait on them to say they got your email.

But if the journal has a good reputation, I would be hesitant to pull your paper. Who cares if they didn't tell you they got it? If they have a reputation for timely reviews, then trust the process.

5:53am said...

5:56/7:28/1:05-

Three weeks seems a bit much. I've had articles rejected in that time. I at least don't see a problem with moving on.

Anonymous said...

November is approaching quickly and the more pubs the higher the chances of getting a TT - so from that perspective, I feel that I have to be pushy.

Otherwise, the journal is well-ranked and I do hesitate to withdraw despite my reservations regarding the process.

So - I'll wait it out and see what happens.

Thanks to all for the advice!

zombie said...

In terms of the job market/your CV, having your paper "under review" at a good journal counts for something. Presumably you have a non-padded section of your CV for things in submission/in progress.

Anonymous said...

@zombie

WHAT?! An R&R maybe, but under review is worth nothing. I mean, I can send a 20-page lorem ipsum to Phil Review right now. Why should that count for anything? I'm dramatizing a little, but only just.

Anonymous said...

@11:24

I think having papers down as 'under review' helps a tiny tiny bit, if you already have publications. It shows that you know what it takes to get published, and are doing it again, and are research-active.

But listing the journals under which it is under review is tacky. We hired this year, and having a 'work in progress' section with several papers under review did show that candidates had a research pipeline.

But I did have a few files where they listed impressive journals where the papers are under review. That counted massively against the file, for me (perhaps irrationally).

zombie said...

11:24: I didn't say it counted for a LOT. I said it "counts for something." It certainly doesn't count as much as having things published, but pulling the paper from a good journal on the chance that you can push it through somewhere else faster seems dicey.
But, all things considered, if someone had nothing published, and something under review, they might not compare favorably to someone who has pubs, or accepted pubs. But compared to someone with nothing at all, better to have something somewhere in the pipeline.

Anonymous said...

@ 1:34:

> But, all things considered, if someone had nothing published, and something under review, they might not compare favorably to someone who has pubs, or accepted pubs. But compared to someone with nothing at all, better to have something somewhere in the pipeline.

No, I don't think so. Neither has demonstrated the ability to do publishable work.

Anonymous said...

Slightly different question, dunno if it deserves its own thread, but:

Short version: Can you cite your own unpublished work?

Longer version: What do you do when you're writing a paper and want to refer to another paper you've written that goes into more detail on a certain point or supporting argument that you don't have time to address at length in the current paper -- but that other paper is unpublished? Are you just screwed?