Why Not License Referees?

The referee system in scholarly publishing offers us many benefits and also carries with it attendant problems. The problems need to be addressed.

Referees are arguably the linchpins of academic scholarship: they do the heavy lifting for editors, they provide editors with vicarious expertise, and they monitor the gateway to publication and thus to tenure and promotion. Their presence in the editorial process is the guarantee to deans and program directors that scholarship is scholarship.

Referees are also, however, the bottleneck of the publication system. Dilatory or slothful referees idly and thoughtlessly put careers on hold.

A by-product of the referee system is that nowadays the scholar never outgrows the status of a graduate student. In a sense, this is good: the great advantage of a professorial career is the concomitant license to be a student all one’s life. But the referee system, in effect, promotes the chosen referee to autocratic graduate committee chair while demoting the author-scholar back to the days of writing dissertation chapters to suit the committee’s approval. The scholar-author was in some ways better off as a graduate student. Though the referee has the arbitrary power of a graduate committee, the referee never has to deal with the “student’s” feelings, and the “student” never gets to discuss the report with the referee.

The system needs changing. People with this much power should be trained. The only requirement for referees is the trust of an editor that the referee is knowledgeable in the subject area. Referee selection parallels the old way of publication where, as recently as the 1970s, authors were published because they knew the editors. In publication, of course, there has been a (mostly) salubrious change. This change has not yet befallen the selection of the referee. Everyone who does it is an amateur at it, and, too often, it shows.

Editors of the journals in every field should get together and establish a training and even a licensing program for referees. Becoming a licensed referee could then count in a professor’s annual reviews.

At a minimum, to begin, editors should establish one rule: there should be a thirty-day deadline, after which the paper would go to another referee. Further, some old rules that used to go without saying must now be explicitly spelled out for a new generation of editors and their referees. For example, just as the chair of a search committee does not get the job, the referee does not become coauthor. The editors could come up with other rules.

They should also establish a checklist for evaluating the article and for rewriting the report so it is fit for the author to see. Finally, another checklist is necessary: since every scholarly author’s professional life is at present at the mercy of amateurs, a checklist must address how to respond to a referee’s report. Every referee response should be the subject of an author’s prompt reply, to which the editor should give a hearing. Both for the editor and the author, allowing a referee report to stand without response is the equivalent of a judge hearing one side of a case, or of a juridical system that lets a witness go without cross-examination.

A referee must think on three levels:

1. Obviously, the referee must consider the article itself (that is where most referees now stop).

2. The referee should be aware of the human being who will read the report.

3. The referee should have in the back of his or her mind that the author, too, has a say and can respond to a review directly to the journal’s editor.

The referee report checklist above is only a stopgap but is a necessary start. In the end, scholars must insist on nothing less than referee certification, like a pilot’s license, with ratings. Until then, we have the virtual equivalent of student-pilots acting as pilots in command, with passengers. The Federal Aviation Administration forbids this because student-pilots who carry passengers put lives at risk. Luckily for us in academe, all that is at risk is our careers.

Tom Winter teaches Latin and Greek at the University of Nebraska–Lincoln and has been in college teaching since 1966. His e-mail address is [email protected].

Referee Report Checklist

1. Preparation. (Some actual work is required here. It is salubrious to remember that in the field of mathematics, referees must work through the mathematics of the article under review with pencil and notepad and, yes, with some concentrated brainpower.)

a. I skimmed the article.

b. I read it through carefully.

c. I read it through twice or more.

d. I have checked my objections.

i. Example: Could the author overcome an objection by simply photocopying the appropriate page of a dictionary?

ii. Or, worse, could the author overcome an objection by simply calling attention to the article itself? (You aren’t still at 1a, are you?)

2. The approach, up or down.

a. Did I learn from reading this article? An answer of “yes” might suggest that the author knows as much as or more on the subject than the referee, should steer you toward approach 2c, and, in the best of all possible worlds, would make section 4, below, completely unnecessary.

b. Does the article state and substantiate a thesis? An answer of “yes” should steer you toward approach 2c.

c. My goal in writing the evaluation is to help make it a better article.

d. My goal in writing the evaluation is to show the editor that the piece is beyond redemption.

3. Writing the report.

a. Have I begun the report by writing a fair restatement of the article’s objectives?

b. Have I evaluated the article in terms of these objectives?

c. Have these goals already been achieved? Remember, a “no” answer here is a positive. This may be difficult; if its job has already been done, the article may seem “correct” and in accord with good doctrine rather than redundant. A “yes” answer should actually steer you toward 2d. In fine, beware of finding yourself asking, “If it’s right, why hasn’t it already been done?” (Yes, a respondent actually said it.)

d. Have I let the article focus on these objectives, or do my requests force the article to veer into other directions? Remember, you are not the author.

e. Are the requests and objections I pose arbitrary? This question is meant as a cross-check on 3b and 3d.

4. Checking the tone.

a. Would I have expressed myself this way to the author’s face?

b. Is the tone of the response written from a point of view of omniscience? This entry could be read, perhaps by a referee, as a desideratum. It is not. “Yes” is the wrong answer to most questions in this checklist section.

c. Is the tone sneering?

d. Is it patronizing?

e. Will any of the observations be taken as an insult?

f. Have I used the phrase “let alone the . . .”?