In my previous post I briefly discussed peer review, which has been raised by many in the publishing industry as a justification for opposing the NIH mandate for deposit of articles into PubMed Central, and, more broadly, as a justification for the vigorous protection of publisher-held copyright in scholarly publications. In this post I discuss the role(s) of peer review in the academy more generally.

Broadly, peer review is the set of mechanisms that enable scholars to have reliable access to the informed opinions of other scholars, in a way that allows that those informed opinions themselves to be subject to similar vetting.

Scholarship requires reliable and robust peer review, and the academy engages in peer review in a variety of ways, both direct and indirect. Peer reviewed publication is one method, and a fairly powerful one at that. If you read a paper in (for my field) Econometrica or the Journal of Political of Economy, you are reasonably confident that accomplished scholars in the field have made a judgment that the paper is of high technical quality and worth reading, and that experienced scholars have made a judgment that the paper is of interest beyond its narrow subfield. Those are valuable pieces of news as one is looking for a way to spend some time, and they also tell you something about the likely quality and accessibility of papers outside of one’s specialty, should one be branching out or needing some background information or trying to figure out who to consider for an open position in the department.

Similarly, the appearance of an article in a leading specialized journal, or of a monograph in a prestigious series published by a scholarly press, conveys valuable information (at least to the cognoscenti in the field) about the quality of the book or paper.

The peers who undertake the reviews are genuine peers. They are scholars whose judgment is trusted by experienced members of editorial boards, who are themselves generally senior scholars in the relevant field(s). Such people engage in peer review pretty much all the time. They go to seminars and talks, read draft manuscripts from students and colleagues, near and far, review grant proposals, engage in workshops, and vet tenure and promotion files. In short, the peers doing the reviews are active scholars engaged in active scholarship. (Sometimes they even spend some time writing their own stuff.) They could no more NOT provide “peer review” then they could give up reading and writing. Peer review is part and parcel of what serious scholars do.

I’d guess (and I would love to see a serious study) that the fraction of time that scholars spend engaged in formal peer review of publications – journal articles and monographs — is less than half of the time they spend on peer review in total. Moreover, the work that has traditionally been done under the aegis of publishers is increasingly being done in other settings. In fields where it is customary to post working papers on the web, interesting papers generate a good deal of peer review in the form of commentary from peers. Given that it takes essentially no time to move from word-processor to web posting, and that it often takes years to get from submission to a journal or scholarly press to formal publication, it’s not surprising that informal peer review is becoming more common. This is good news. Scholarship advances more rapidly if work can be widely shared relatively quickly and easily. Given that publication in the literal sense (making public) is now easy and cheap in the technical sense, it seems almost certain that informal review will grow relative to formal review.

For several years, I was the chief academic officer of the University of Michigan, and I have been involved in the review of tenure cases, grant proposals, journal articles and book manuscripts for more than 30 years. The most interesting and important of these activities are reviews associated with tenure and hiring. It is often argued (quite explicitly so by some) that without the reviews associated with publishing, the academy would be at a loss in making judgments about the quality and productivity of scholars. To be sure, for reasons adduced above, a record of publication in strong peer-reviewed settings conveys valuable information to tenure and search committees, chairs, deans, and provosts. But the fact of the matter is that we pay equal attention to other reviews, including (for some fields) those required to obtain research grants, and (for some fields) post-publication reviews that appear in journals and other venues. We also take very seriously the opinions of ad hoc reviewers, inside and outside of our institutions, who prepare and evaluate the case for promotion and hiring. Take away the information conveyed by publication venue, and these tasks become more difficult, to be sure, but by no means impossible. And the essential part – close reading of the work by peer reviewers – remains intact.

Just as it pays for almost all of the content that goes into scholarly publication, so too does the academy – colleges, universities, research centers, and the entitites that fund them – pay almost all of the costs of peer review.

Publishers provide many useful services, but they do not provide peer review. It is the peers themselves who do that essential work.