Thursday, May 23, 2013

R for the R & R (Requiem for the Revise and Resubmit)

It occurs to me that it has been something like five years since I've had a "revise and resubmit" verdict from a journal. In that time, I've had several papers accepted "as is," several more conditionally accepted as long as I am willing to make the changes and successfully execute, and a bunch of rejections. Many of these rejections, of course, come with no comments. But when they do come with comments, it sometimes seems to me that the actual comments are basically consistent with an R&R. They say, "a few minor suggestions," or some such thing, not, "there were several serious problems." Often these reports don't say anything specific about what they recommend the editor do—that material usually goes straight to the editor—but one report I recently got was explicit that the referee thought the paper should be R&R-ed. The editor, of course, rejected the paper.

And it seems to me that many of my friends have had the same experience. At least, the ones I've discussed this with. Although this is so unscientific that it's of basically no value whatsoever. Probably shouldn't have mentioned it. 

But I have gotten the sense that, over the past ten years or so, space in the journals has gotten increasingly scarce—witness Nous and PPR's annual six-month submission hiatus—and that this makes editors increasingly reluctant to deploy the R&R, and to reach straight for the "reject" button instead. 

And that's too bad. I like R&Rs, and not just because it's not a full-on rejection. I like hearing that the editor believes in the paper enough to give it another shot. I like getting real feedback from an editor who isn't sending it along just in case I'll find it helpful, but who actually believes that the suggestions will make the paper better. And I like being in a position to discuss and possibly negotiate the proposed changes. I think there's legitimate value in the R&R, and it makes me sad to see it go. If, indeed, it is going. Which, maybe it's not. 

Does that seem right? What am I missing? 

--Mr. Zero


Anonymous said...

I've gotten a few papers like this (R&R-style comments on a rejection). But to be perfectly honest, here's something I've noticed much more often: my papers are getting rejected for extremely nitpicky reasons. I have the impression that the flood of submissions is so great that referees are looking for any reason at all to reject a paper. And I don't think it's helping the profession.

My impression is that when you raise the bar this high, the only papers that make it to publication are ones that are technically precise, draw too many distinctions, and add little to the literature beyond what has already been said. In other words, by being extremely selective you actually end up with fewer interesting papers.

Hell, maybe I'm wrong. But I suspect I'm not.

Anonymous said...

R&R is nice, and my R&R'd papers have always substantially improved, and always been accepted (so far). I continue to get R&Rs though, unlike you, Zero. But from 'top-10' journals I also get comments that look like an R&R verdict, with a rejection decision by the Editor. I get the impression that at such places they look for every reason to reject, which is understandable in some ways.

Kind of sucks though that one reason to reject is 'I don't know this person or their school.'

Just kidding.

Unless that's true.

Mr. Zero said...

here's something I've noticed much more often: my papers are getting rejected for extremely nitpicky reasons.

Yeah, I've noticed the same thing. And I share your concerns about it. It's for shit.

Anonymous said...

Two comments, if that's ok:

1. Zero, I suspect you may be right (based on my limited experience and what impression I've received from others). I recently had a paper returned to me, rejected, from a mid-tier (but highly-specialized) journal. The reviewed recommended rejection. The editor, however, informed me that while was was "officially" rejecting the paper, she suggested that I revise to address the reader's (valid and helpful) concerns. She noted that she thinks I should revise the work, and to consider that journal again, after such revisions are made. It all sounded very convoluted, and I now wonder if perhaps the journal policy is to officially reject everything not immediately accepted, but she was trying to suggest R&R to me, even if unofficially. Seems silly, but I read her email now as a nod to official policy, while recognizing the spirit of R&R.

2. To 10:58, I agree that such practices exist, and that they are not helping the profession. However, the obvious solution is one that very few established faculty are willing to accept. In this day and age, there is no reason why we need physical journals anymore. We just don't. We can easily print .pdfs if we want hardcopy, but no journal needs to put together and mail out a physical product anymore. And the way too many in the profession fetishize the "print journal" and demonize "online publication" is simply absurd. We could blow open the doors, and have online publications that work to expand what we know, that aren't afraid to take risks, etc., for a fraction of the cost (said cost, I feel, being a primary reason why many journals won't take big risks; too many risks, and too many failures, and the subscriptions drop).

Unfortunately, the single biggest stumbling block here is the field itself. Established faculty (many, anyway) refuse to accept the legitimacy of online-only publications (as if "print" somehow equates to "thoroughly reviewed," forgetting that until the actual issue is printed, all the reviewing is done via electronic copy anyway). Younger scholars (grad students up through untenured junior faculty) can't afford to publish outside those established venues, given the state of the market. The field loses *nothing* by opening up publishing. The bad work will, like bad work before it, be lost. It will go unread. The good work will be kept, will be read, will be taught, will be cited. It will add to the discussion.

Let history be the judge of what is valuable, what has merit, what contributes to the field (which is something we never know upon first publication anyway). The current publication model is nothing more than nit-picky gatekeeping, and doesn't serve the needs of the profession. Yes, once upon a time, print journals were a wonderful way for scholars to hold professional conversations at great distances. But we no longer need to be tied to a publication model that's a century old. Lord knows, I also don't cook by building a fire in my kitchen stove, or head into town by hitching the buggy up to the horse.

Anonymous said...

Unfortunately, the single biggest stumbling block here is the field itself. Established faculty (many, anyway) refuse to accept the legitimacy of online-only publications (as if "print" somehow equates to "thoroughly reviewed," forgetting that until the actual issue is printed, all the reviewing is done via electronic copy anyway).

I literally have no idea who you're referring to. Philosophers' Imprint, Journal of Ethics and Social Philosophy, and Semantics and Pragmatics are all online, open-access, and extremely well-respected. All those journals also have editors and boards that are well-respected in the profession, and gains legitimacy that way. Their online status is absolutely irrelevant.

What is true is that Joe Schmo cannot just go start an online journal and expect it to gain recognition. But, again, the issue has nothing to do with online vs print. The reason that such a journal would gain no cred is that Joe Schmo don't got none.

It's hard to start a journal. It's even harder to confer legitimacy on a journal. All that makes for a slow process. But the slowness has nothing to do with the alleged old fogeys' disdain for the internets.

zombie said...

I just got an R&R (with quite good comments, and a very nice letter from the editor) not two weeks ago. I think it's my first (previously, papers were accepted with some revisions, or rejected outright).

But I was recently told by an editor of a specialty journal with Springer that Springer is now insisting on a significantly lower acceptance rate (although I was not told why).

3:11 - are you suggesting that by going digital only the journals would publish more papers? What makes you think they would do that?

Jamie Dreier said...

Zombie, online journals can publish more papers because the cost of publishing one more paper is practically zero.

I'm one of the editors of JESP. As a matter of fact, we do try to have a very high standard and encourage our referees to do so too, but I'm certain we publish papers that other journals would reject for (as others have put it) nit-picky reasons. The general idea among the editors is, we want our readers to feel like they can count on papers published in (at?) JESP to be worth reading, but obviously we never worry about space considerations.

I wish I had some insight into the issues raised in the OP, by the way, but I just don't know the answer. The hypothesis (that R&R verdicts are down because editors have to be more selective) seems very plausible.

Anonymous said...

"my papers are getting rejected for extremely nitpicky reasons"

Same thing.

I'm getting really annoyed with the way Nous and PPR is being run. These are really important journals for our field and there shouldn't be one person in charge of both.

zombie said...

"Zombie, online journals can publish more papers because the cost of publishing one more paper is practically zero."

Jamie, I'm aware that the costs would be somewhat less for digital-only publishing (although I doubt it's zero, since copy editing, website maintenance, and other professional services would still be needed). But I wonder why it should be assumed that just because the journal *could* publish more without incurring much greater cost, it *would* publish more.

There are many things one could do, but that does not make it the case that one would do such things.

Jamie Dreier said...

Right. Well, I said the marginal cost is *practically* zero. There is some copyediting and formatting, true enough.
My only point, really, is that insofar as the refereeing process has become tighter and maybe pickier *because* journal space is tighter, online publication should be relatively free from the pressures. That is, if there are papers that would be published but for the journal space crunch, online journals are a lot more likely to publish them.

Anonymous said...

I've never worked for a journal, and certainly I have never started one. But what, exactly, does it take?

Assuming a comparatively low overhead for an online-only journal, what does it take to launch a new journal? An editor and an editorial board, all of whom should be respected enough in their fields to give the new journal some legitimacy at first. Let's assume that the editor negotiates dedicated server space at her university, and possibly graduate assistants to help with the operations of the journal. (This, I imagine, is where some of the cost comes in, as those students need to be paid.)

The journal would also need readers for the articles, some of whom might be on the editorial board, some of whom will be other recognized scholars. It might be tough to convince some people to take on this work, especially if they already read for other journals, but I don't see where the field is lacking for qualified readers. In fact, some may jump at the chance, as it might be good for their own CVs (in terms of tenure and promotion).

I know it can't be this simple, so what am I missing? Yes, the need for someone to design and maintain the website, and that costs money, too. But what else?

Mr. Zero said...

Hi 11:44,

You'd need all that and the services of a copy editor. And my impression is that, at least in professional applications, server space is one thing and you'd have a separate charge for data transfer. (This doesn't come up in rink-a-dink blogger blogs, so it's possible that I don't know what I"m talking about.) Maybe some other stuff, too. If Prof. Dreier is still following the discussion, he probably has some idea of what he's talking about.

And I think you're being entirely too sanguine about all of this. Each of these necessities constitutes a significant problem. From finding competent referees who submit their reports on time seems to be really tough, even for established operations--witness any discussion of journal referees that has ever occurred anywhere in the world. Server space and data transfer are expensive--last year Philosopher's Imprint considered charging a submission fee in order to defray costs. (Discussion here.) I think that getting a journal started, even if it were all-online, would be a ginormous amount of work.

And, it would need to be crystal clear from the get-go that you, your journal, your backers, and the infrastructure you set up to support it were in it for the long haul. A journal cannot be the little engine that could, who definitely thinks it can make it, but everybody knows has a 50/50 shot. (Please pardon this horrible metaphor. I have a small child.) Nobody wants their work to appear in a journal that might not be there in a few years. Especially an all-online journal, which, unlike a print journal, could just evaporate into the ether without leaving any trace of itself. What I'm trying to say is, there would be serious trust issues you'd have to overcome.

So, I mean, you could do it. People have done it. It's doable. It's possible to overcome the challenges. But I think it would be pretty hard.

Dr. Killjoy said...

While I do think elevated standards to blame for much of this, folks should also remember that often the referee comments they receive from the editor may not accurately reflect the entire story. For example, suppose you receive a rejection even though the comments seem to straightfowardly suggest a R&R. It may well be due to increased standards, but it could also be the case that there were two or more referees. The one recommending R&R provided most if not all of the comments sent to you by the editor but the other(s) recommending rejection provided few if any useful comments for the editor to send. Furthermore, editors often select comments from various referee reports they think most helpful to authors and as a result comments that seem in-step with an R&R might be nothing more than the cut and pasted comments from two or three referees all of whom recommended straightforward rejection.

Again, I think increases in standards in the face of submission pressures might explain a fair amount of this phenomena. However, so too might the relatively shitty quality of the papers. The minimal duties of the referee do not include pointing out the various ways in which authors could revise their papers into things of publishable quality. As such, perhaps we ought not assume that their doing so itself somehow warrants a decision other than rejection.

Jamie Dreier said...

Mr. Zero,
Years of experience in academia have taught me it’s best to avoid knowing much about finances.
I do know that JESP is fairly expensive, and would not be possible without significant contributions from Annenberg, USC College of Arts and Letters, and USC Law. I believe the main expense is our paid Managing Editor. (The same is true of ETHICS, by the way – a full time Managing Editor, but of course that journal makes tons of money.)
You are absolutely right that an essential feature is a rock-solid and committed institution behind you to guarantee that the published work will be around indefinitely, and enough gravitas in your collection of editors to encourage contributors to believe (and to believe that others will believe, and to believe that others will believe that others will believe, etc.) the enterprise will be successful enough to make it worth their while to contribute.
JESP was lucky to get Raz and Dancy to contribute papers very early, plus a bunch of good stuff from good people who weren’t superstars. And look at the first eight or so articles in Philosophers’ Imprint. They left no doubt that publishing there was *serious*.

Anonymous said...

Mr. Zero, one thing that might explain *some* of what you observe is the new special strict instructions that some journals are giving to referees. For example, along with a recent paper sent to me by Phil Studies were instructions that said something like: "Given the backlog, please be extra strict and use super high standards" (I can't recall the exact wording). With instructions like that, you increase the likelihood that the papers that aren't rejected are good enough to be accepted or conditionally accepted. Thus, this kind of instruction might explain some of the disappearance of the R&R (if, indeed there are less of them). From my own personal experience, I've had a lot of R&Rs lately and rarely if ever receive the cherished "straight up acceptance." If you want R&Rs, I'll trade you some of mine for your acceptances!

Anonymous said...

I had an R&R two years ago from a journal with very low acceptance rates. FWIW

Anonymous said...

My recommendations as a referee has almost always been either reject or R&R. It's very rare that I get a paper that doesn't need some substantial work. I imagine my reject comments can look a little R&R, as I do usually point out the problems in the argument, and sometimes how I think they might be fixed. Of course, whether I give an R&R or not doesn't mean that the paper gets what I recommend (other referees, editor, etc). I also sometimes write to the editor in the separate "confidential" part of the recommendation that, for example, I suggest reject but that I'm close enough on the scale to be fine if the paper gets an R&R, if the other referee likes the paper a lot (and, obviously, I say more about why).

Recently I've been getting more and more papers that are overshooting the journals - perhaps it is the pressure of the ridiculousness of the job market? Many of the papers I've been reviewing for journals in the top tier really aren't anything more exciting than a mediocre grad student seminar paper. And it's also sad to get a paper that is an ok paper in most respects, but that really is nowhere near the quality for Mind, etc. Especially when you know it may have been sitting on an editor's (virtual) desk for months. Don't waste your time on those top journals (even if they have a fast-ish turn around) if the paper doesn't add a substantial point to a debate in an interesting way. Maybe, if you can, find a mentor who can help you place your papers in appropriate venues? Of course, it may be worth overshooting a little, but if the paper is clearly not substantial or interesting enough for the journal, it will be rejected, and that's time that the paper could have been at a journal that had more chance of accepting it.

Anonymous said...


I think there are several reasons for what you have seen:

1. Some students are sending work to journals knowing that the best they can hop for is R&R, and basically want a professional reader to provide useful feedback. (I know some faculty who advise their grad students to do this.)
2. Some faculty have stopped publishing, and are not up-to-date with their own reading, and so aren't as familiar with what journals are publishing now, or what the standards are. (This includes some faculty at ranked institutions, who earned tenure many years ago.)
3. Some students are not as good as others at figuring out where their works are, in terms of development and clarity. There are many people with excellent ideas, but terrible execution. Similarly, there are many people with excellent execution, but mediocre ideas.
4. The market. Yes, always the market. And I also know some faculty who advise students to send out works the year they go on the market, so they can list the papers they have under submission at various journals. And of course, if you are going to do that to create a line on the CV, you're going to send to top journals. Best-case scenario: you get an acceptance or an R&R; worst-case scenario, you get rejected, and can then blame the rejection on the journal's shrinking acceptance rate.

Anonymous said...

@11:44 Here are the main problems, as I see it:

1. If your journal does not have a very reputable editor (like Darwall, Velleman, Dreier, von Fintel level reputability), I will not submit to it. The fact it, there are few philosophers who have that level of reputability. Fewer of them want to run journals.

2. If your journal does not already have some reputable scholars publishing in it, I will not submit to it. I want my work to be read, not just published. How will you attract those reputable scholars? Usually, the reputability of the editor is crucial in building that initial batch.

3. If your journal does not have a permanent archiving solution, I will not submit to it. Getting some web server is far from adequate. Who will run the web server after you die / lose interest / etc.? How will the articles remain accessible even if the journal folds?

4. If your journal does not have a permanent identifier solution, I will not submit to it. How will other scholarly works refer to articles in this journal? A standard solution will involve DOIs.

Semantics & Pragmatics had a blog documenting all the hard work that went into starting that journal. It'll be instructive.

Charles Pigden said...

Since R&R tends to mean rejection I try to avoid it. Instead, if the paper is basically OK, I recommend acceptance, adding some 'comments that that the author might like to consider' but making it clear that publication should NOT be conditional on meeting my criticisms.

I recommend this policy to other reviewers. It means that more good papers will be published quickly though it also means that they will often not be as good as they might otherwise be.

Anonymous said...

"I recommend this policy to other reviewers. It means that more good papers will be published quickly though it also means that they will often not be as good as they might otherwise be."

So we sacrifice quality for speed and volume?

Anonymous said...


I don't think I've ever seen a DOI used in the citations/bibliography of a philosophy paper or book. DOIs certainly have their uses, but allowing 'other scholarly works [to] refer to articles' doesn't typically appear to be it, at least in this discipline.

Anonymous said...

The big problem, it seems to me, is jointly that referees often don't know the difference between "reject" and "revise and resubmit," and some journals (those that typically strive for >90% rejection rates) are treating R+R judgments as rejections (or explicitly encouraging referees to find any reason to reject a paper).

With that said, I have received R+Rs this past year form a number of top journals (5 in the past 12 or so months, in fact).

Anonymous said...

referees often don't know the difference between "reject" and "revise and resubmit,"

-Do you think you could elaborate on this?

Curious said...

Before speculating on *why* R&R verdicts have declined, can we get data on *whether* they have declined? (Doesn't ETHICS, for one data-point, publish this sort of info every year?)

Anonymous said...


I often see referee reports that recommend rejection, when they're really revise and resubmit.

What I mean is that referees often don't know what they should recommend for rejection, and what they should recommend for R+R. There's no real surprise here: (essentially) none of us receive *any* training on how to review articles for journals, how we should make our decisions, and what we ought to recommend.

Anonymous said...

For 2010 and 2011, 81 of the 900 or so papers submitted to Ethics received R&Rs. About half of the papers at Ethics are rejected without being sent out to referees, so of those that go out to referees, so about one in five of the papers sent out to referees received an R&R.

Anonymous said...

"none of us receive *any* training on how to review articles for journals, how we should make our decisions, and what we ought to recommend."

And yet you feel comfortable suggesting that you know better? Because of the objectivity of a "because I said so" claim?

Anonymous said...

my papers are getting rejected for extremely nitpicky reasons. I have the impression that the flood of submissions is so great that referees are looking for any reason at all to reject a paper. And I don't think it's helping the profession.

Of course referees and editors are looking for reasons to reject, due to the surplus of submissions. That's no secret; it's announced to referees as a goal. Not sure what you're suggesting would be better for the profession, but more journals publishing more papers doesn't seem like a good remedy. Most papers don't get read as things stand.

Also -- though I of course have no idea about your papers -- a rejection for "nitpicky" reasons is entirely compatible with this sort of report:

Comment to the editors: "This paper lacks sufficient originality and interest to warrant publication."

Comment to pass on to author: Insert nitpicky criticism here.

Why wouldn't the referee want the devastating comment sent along to the author? Because it wouldn't help the author improve the paper, merely hurt morale; and perhaps another referee, especially for a lower-tier journal, will see something to like about it.

The Brooks Blog said...

R&R is very much alive at the Journal of Moral Philosophy: don't believe our acceptance or R&R rates have changed much over the past few years.

Anonymous said...

Written at the time of George Jones' death:

He Stopped Publishing Today

He said he’d publish or he’d die—
She said resubmit, revise—
As the versions slowly climbed—
The rejections massed sublime—
They claimed each fixture of it flawed,
A flaccid maze of argument,
He resubmitted through it all
Claiming each point rebutted—

Rejection letters often fed
Into his inbox—not a few—
He saw underscored in red
All the noted “please review”—
I saw his CV just today—
Oh but I didn’t see no fears
Of a reconsidered say
On a paper out two years—

He stopped publishing today—
They stamped “rejected” and he saw
That more were coming on the way—
He stopped publishing today.

She sent fresh edits from her Mind—
Oh, and he wondered if she would—
But just counterfactually he’d find
That they were usefully much good—

He stopped publishing today—
They stamped “rejected” and he saw
That more were coming on the way—
He stopped publishing today.

Anonymous said...

One other reason why journals, online or print, don't publish more articles and are less picky: being accepted by a journal with a higher acceptance rate is more impressive in a job or tenure application, which makes submitting to such a journal less desirable. And since submitting to a more lenient journal is less desirable, such a journal is less likely to get the interesting thoughtful articles in the first place.