Nota publicada por Dan Greenberg en Brainstorm, el el Blog del Chronicle of Higher Education, 26 noviembre 2007
Texto a continuacion.
Does Peer Review Pass Peer Review?
The adulation of peer review in the popular press is one of the great wonders in relations between science and the rest of the world.
For attesting to the credibility of a scientific report, news accounts routinely note whether or not publication has occurred in a peer-reviewed journal. If it has, the legitimacy rating goes up; if it hasn’t, a presumption of doubt is overtly or subtly conveyed. The actual workings of peer review are so little known to the public that a street-corner survey would probably turn up hunches that it has something to do with Coast Guard inspection of waterfront facilities. Within science, the common defense of peer review echoes Churchill’s accolade to democracy: the worst possible system, except for all others. Basically, it consists of pre-publication screening of reports by presumed experts in the subject matter. Their candor is supposedly assured by a guarantee of anonymity, though the grapevine crackles with reports of scores settled in the course of peer reviewing. Thefts of hot ideas are also said to occur behind the curtains.
Failings of peer review rarely come to light, but when they do, the basis for confidence in the ongoing system looks shaky. The South Korean cloning fraud cleared peer review en route to publications in Science in 2004 and 2005. In the 1990s, as the so-called science wars raged between traditional scientists and social-constructionist social scientists, a hoax — celebrated by the former, damned by the latter — was pulled off by a mainstream physicist, Alan D. Sokal, of NYU, author of a turgid, nonsensical paper: “Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity.” Submitted to the journal Social Text, the paper was revised at the suggestion of peer reviewers, and then published. Snookered journals generally respond in the same fashion, announcing a review of their peer-review procedures, while insisting that peer review cannot cope with malice, criminal intent, or other deviations from principled behavior.
Unrewarded by anything more than a thank you, peer review provides expert free help for editors confronted by avalanches of papers. They need it as a first line of defense against sloppiness, triviality, and other failings in science, even if it’s a porous first line. In the absence of any other accepted system, it’s the gold standard of scientific quality.
But even seasoned editors acknowledge the weakness of that defense line. Consider the lamentations of Drummond Rennie, an editor of The Journal of the American Medical Association, who’s a leading campaigner for uplifting the quality and integrity of the scientific and medical literature. Writing in 1986, Rennie observed that among peer-reviewed journals, “There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled … no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print.” To which he added, “But an unbiased reader, roaming at random through a medical library, would find in abundance all the problems I described in 1986.”
Mindful of the limitations and criticisms of peer review, the journal Nature experimented in 2006 with “open” peer review, inviting prospective authors to post their submissions on a publicly accessible site for evaluation and comment from any source. Few took up the offer, and few of the posted papers drew responses, leading to a termination of the experiment.
So, the peer-review system remains little changed, except for increased insistence on reviewers disclosing possible conflicts of interest — a requirement that’s unevenly applied and sometimes ignored. Anonymity of reviewers remains enshrined in the system, even as “transparency” is widely lauded as a guarantor of upright behavior in academic and scientific affairs. In the popular and scientific press, book reviewers are identified, without apparent loss of candor. But science won’t risk it.
Remedies such as open posting of submissions and identification of reviewers draw little support in the scientific community. Peer review doesn’t fare too well under peer review, but for the popular press, it remains a certification of scientific quality, though most scientists know better.
Sobre el autor
Dan Greenberg is a longtime observer of science policy and politics. He is the author of “Science for Sale: The Perils, Rewards, and Delusions of Campus Capitalism” (University of Chicago Press), as well as other books, and has published widely in newspapers and popular and professional magazines.