Peer Review

        Are you puzzled by this term?   I'll admit, it's a curious formula.  However, it makes all the difference in the world when deciding whether and why to trust a source.  The term "peer" comes to us from medieval English law where it was used in Anglo-Norman to refer to the barons who ruled England and whose competition for local power was kept in check by the royal government if it was strong and wise enough.  We are familiar with the word from the Magna Carta (1215) where the right to "trial by a jury of one's peers" meant that a baron would be tried by other barons, not by a court packed with hand-picked supporters of the crown.  The "peer-reviewed journal" or press uses the term, only slightly ironically in the democratic cultures of England and America, for a process by which publication should be proof that a just trial of the work has taken place and it has not been "imprisoned" without publication nor "set free" in print because of any unfair prejudice on the part of the press's owners. 

        The "peers" who review articles for journals or books for university presses, the "barons" of higher education, are the competitors and colleagues of those who write the articles and books, other practicing scholars with earned degrees, most of whom hold teaching posts and all of whom publish on the work, author, era, and/or phenomena with which the article or book deals.  That tension between competition and collegiality produces the peculiar mixture of acute attention to detail and logic combined with respect for another's right to differ reasonably which makes a well-run peer review possible.  If the reviewer doesn't really compete with the author for the power to interpret that kind of evidence, the review may be ill-informed or careless, too easily impressed or inaccurately finding fault.  If the reviewer does not maintain a professional respect for the author, the review can be fatally biased.  Good editors assign work to peer reviewers to prevent those kinds of errors, though occasionally they submit a work to a reviewer who is known to disagree on principle with the theory underlying the work's interpretive practice--a strategy for determining how "bomb-proof" the author's use of evidence and logic elsewhere has been.  A hostile but principled reviewer will dig out any scrap of error to be found, and that's what a good scholarly press wants done, painful as it is when it happens.  This is all in aid of the basic principle of scholarship: Tell the truth.  With a principle like the truth at stake, many sacrifices are possible.

        Here's how the process works.   When scholars complete manuscripts they believe are good enough to be published, they decide on the proper journal or university press and consult the style sheet to put the manuscript into proper format.  Usually, peer-reviewed journals and presses request multiple copies of the manuscript for rapid distribution to their stable of peer reviewers, the scholars they have asked to read manuscripts and to render an opinion about their "publishability."  Click here for a sample peer-reviewer's form, read pages 6-9, and think about the questions reviewers are being asked about the work submitted for publication. 

        "Blind" peer review is the most scrupulous form of the process, in which the manuscript's author's name and institution appear nowhere on the page so the reviewers may be reading the work of a newly-minted Ph.D. or a senior scholar with decades of publishing experience.  (Stanley Fish famously denied the value of blind peer-review in a PMLA article, saying intelligent exercise of senior scholars' prejudices was preferable, but the matter is still in debate and, perhaps not coincidentally, PMLA recently has had so few submissions of unsolicited manuscripts that hardly any of its articles are now peer-reviewed, at least since late 2000--they have to ask scholars to write for them!)   Typically, reviewers are never identified to the MS authors, to protect their ability to express honest judgments without fear of reprisals later in their careers from an offended author.  The editor collects the returned peer-review reports and forwards them to the author with a summary recommendation of three types: the MS is ready for publication as it stands (very rarely); the MS is suitable for publication if certain questions can be answered and certain errors can be corrected; and the MS is not suitable for publication.  Lately, a fourth recommendation has appeared as journals and presses become increasingly specialized: the MS may be suitable for publication but does not fit the particular focus of the journal or press.  Only in the first two instances will the MS reach print, and in the second it will undergo yet another round of scrutiny by scholars before typesetting.  That is what makes peer-reviewed scholarship immeasurably different from things printed in weekly or monthly popular magazines that only have limited "fact-checking" departments.  They are completely different from self-produced Internet sites which, unless they are routinely inspected by the author's "peers," have roughly the same level of scholarly authority as graffiti on a bathroom wall.

        Is the process fool-proof?  No, of course not, but the "tell the truth" rule guarantees that peer-reviewed scholars' facts will be checked and their arguments will be weighed for logical consistency long after publication.  If substantial errors of fact or reasoning are detected in important scholarly work, you can be sure their detection will make huge waves in all later scholars' work.  For an important recent example of what appears to have been misrepresentation and raw invention of evidence by a tenured scholar, go to Google's search page and copy into its search window this string of words: Bellesiles gun control  Because the penalties for violating the rules are so high, errors in modern peer-reviewed scholarship tend to be small and easily correctable.  Even the peer-reviewers who approve manuscripts which contain serious errors will suffer damage to their reputations. 

        For an interesting example of a very careful, very fortunate reviewer who sensed the errors of reasoning in Bellesiles' book even before the falsified data was discovered, see Daniel Justin Herman's review on H-Net, a valuable online scholarly review source for books in Humanities and Social Sciences.  Just enter "gun control" in the search topic window on the left and scroll down to "Herman on Bellesiles."  Herman's review was posted in May 2001, and the first popular press articles announcing discovery of the deliberate deceptions didn't appear until October of that year.  Had they been just a bit more suspicious, the pre-publication peer-reviewers might have saved Bellesiles, Alfred A. Knopf Inc., and the gun control movement a great deal of damage.  As it stands, Herman's sober, scholarly assessment of the flaws in the book's use of evidence would have been enough to warn you not to use it without careful thought.  This is the real value of reading book reviews for works which you depend upon in your own writing.  Listen to your colleagues and let them guide you.

        Now that you know how the quality of scholarship is protected by peer review, click here to read about when and how scholars use their peers' publications to help their own published research findings.