Tuesday, June 23, 2009

Helping The Journal Wiki

In comments, chrono asks,

how can we make the journal wiki not suck?

I don't know. One thing that would be nice is if there were a grid. That way it would be easy to tell how long this R&R took, and this A, etc. Wouldn't it be easy to make some kind of an HTML grid for each journal, and then add a column whenever you update it? And it would be nice if there were some sort of "average" function, that would tell you how long, on average, an initial decision takes. I know you can kind of eyeball it, but real math is better. I know I could do it myself, but it has always been my view that real math is best done by others.

What else would make the journal wiki not suck? And Chrono, if you wanted to make some specific remarks about what you think sucks about it, that would be cool, too.


Anonymous said...

It would be easy enough to do, if someone converted the page to a structured wiki:


Doug Portmore said...

I started the wiki. At the time, I wanted to use a wiki spreadsheet but couldn't find any free wiki spreadsheet. And admittedly there are number of other ways the wiki needs to be improved. If someone has the time to look into a better alternative and will do the eventual transfer, I would be happy to publicize the move via PEA Soup.

Anonymous said...

About the problem of journals not communicating well to those who have submitted articles for review: I have a question. If the journal does not acknowledge that it has even received your article for review, would it be unethical to just submit the article to another journal? In this way, there is a legitimate reason for having two journals look at your piece at the same time (and, in a sense, beating the system). If, the journal that didn't ever acknowledge your submission but was processing it in fact accepts it, and the other journal also accepts it, you have an excuse for pulling the plug on the first journal. What do you think?

Peter Bradley said...

The effort towards more transparency in the discipline evinced in this discussion reminds me to announce / promote my new word count 'Trends in Philosophy' tool. I got sick of people spouting off about the discipline with only anecdotal evidence in support, so I built a little script that pulls RSS feeds of journals and counts the frequency of important words in the abstracts. The results are published as an RSS feed. The feeds are updated daily, following an update to the journal feed. The site is at: http://inquiry.mcdaniel.edu/trends/

Xenophon said...

Peter Bradley, that's very cool. Thanks for doing it. I'm wondering what we can conclude from it, however, that's at all useful or relevant to the question of "recent trends" that you say motivated you to do all this work.

If the RSS feeds only contain info on one or two issues, your n will always be small (or do you have a database so that over time the n will grow?).

Is it a trend that the two most common words in the Journal for the General Philosophy of Science are "scientific" and "science"? or that "something" occurs five times in Philosophical Review?

I'd think the most useful result is that "Kantian" and "duty" appear frequently in Philosophy and Phenomenological Research and not, say, "virtue," but I'm hard pressed even here to see how the inference from this to any useful claim about trends is warranted.

Individual keywords just aren't robust enough to answer the question that was posted on Leiter's blog.

chrono said...

The idea of a journal wiki is great. And there's a good chance the wiki in its current form was necessary in order for there to be a better one in the future. But I hope the future is not too far off.

Here are the problems I can think of. 1) Too many people don't post. This has a lot of consequences, the foremost being that it's not at all a representative sample.
2) People don't post full details. Take JPhil. There are 21 "initial decisions" and 18 "final verdicts." So, we can't figure out how specific initial decisions line up with final verdicts. So, we can't figure out how long a certain final verdict took, how many R&Rs end up being accepted, and other such things.

It seems, then, that only thing the wiki is useful for is average time to initial decision. There is no way to get a sense of % of initial acceptances, % of R&Rs that are ultimately accepted, or % of submissions ultimately accepted.

I can't figure out how to solve these problems. But I would hope someone out there can. I thank Doug Portmore for starting it, and I hope that his idea can end up giving birth to something amazing.

Popkin said...

2) People don't post full details. Take JPhil. There are 21 "initial decisions" and 18 "final verdicts."

Isn't that because if the initial decision is rejection then there just isn't a final verdict?

It seems, then, that only thing the wiki is useful for is average time to initial decision.

The other thing I find really helpful is the # of referee reports provided. It allows me to avoid journals that make a habit of rejecting work without getting comments to you.

Xenophon said...

Let me combine two themes that have been running here recently. The other APA (for classicists) has a committee that bugs the hell out of journal editors for data, and then they publish the results. So you know the percentage of papers accepted, acceptance rates for women and minorities, etc. I don't remember whether they get time-for-response data, but they certainly could.

Moreover, our APA could do likewise, if only the APA DIDN'T SUCK. The journal wiki is incomplete but doesn't suck, whereas the APA . . . well, enough said.

The journal wiki will always suffer from selection bias (chrono's complaint) as is: we need an organization with clout to collect systematic data.