Spiros's challenge is to find objections that do not fall into four broad categories:
1. Objections based on a mistaken characterization of what the PGR is (its methodology, how it is produced, what it aspires to track, etc.). (E.g., "The PGR is just a small group of Brian Leiter's friends desperately trying to uphold analytic orthodoxy in the profession" -- actual quotation, by the way.)
2. Objections, also based on a mistaken characterization of what the PGR is (and its objectives), that claim that the PGR fails to satisfy its own objectives. (e.g., "The PGR, being just a small group of Leiter's friends, can't possibly be an objective measure of actual faculty quality" -- actual quotation,)
3. Objections to the effect that the PGR is harmful because it is too easily misunderstood/misused by faculty, students, and administrators.After an hour or two of looking, I dug up the links below. Note that I intend these links to serve only as a response to Spiros's so-called "PGR Challenge."**
4. Objections to the very idea of surveys / rankings / reports of the kind that the PGR is.*
Richard Heck's original criticism, courtesy of the Wayback Machine (via Heck's current website).
Zachary Ernst's 2009 critique, "Our Naked Emperor."
The Smoker's own Mr. Zero's "PGR Minutiae" and "Bride of PGR Minutiae."
Some entries at Choice and Inference on the PGR's "sampling problem" and the "educational imbalance within the PGR evaluator pool." There are also many other posts linked within these on the Choice and Inference blog.
Jennifer Saul has a post at Feminist Philosophers to her paper “Ranking Exercises in Philosophy and Implicit Bias," which appeared in Journal of Social Philosophy, 43:3, 2012.
Alan Richardson has a brief discussion of the PGR in his 2012, HOPOS, 2:1 (1 - 20), "Occasions for an Empirical History of Philosophy of Science: American Philosophers of Science at Work in the 1950s and 1960s." (I highlighted the discussion with screencaps on Twitter [the last three or four tweets]; the editor of the PGR calls this strain of criticism a serious objection in the second link at the top of the post.)
I welcome any further links or examinations of the objections in the comments below.
-- Jaded, Ph.D.
*Spiros calls objections 1 and 2 obvious failures; 3 is not an indictment of the PGR, but of the reading comprehension skills of the various parties (and any such consequentialist arguments, he states in the comments, are failures because they don't consider that the positives, e.g., more information for grads, might outweigh the negatives, e.g., (my favorite) conservatism); and 4 fails since we all "walk around with some such reputational ranking of various programs."
**I leave it up readers to determine if they fall into the above four categories, are successful objections, etc. (I should note that I'm partial to the conservatism worry, as I mention at Spiros's original post.)