El debate sobre la efectividad y la justicia del procedimiento de revisión por pares (peer review) como paso previo a la publicación de artículos en revistas científicas (o para la selección de proyectos de investigación a ser financiados) ha merecido múltiples críticas últimamente y sugerencias para su mejoramiento.
Un ejemplo de dichas críticas se contiene en el artículo de How to Fix Peer Review de David Kaplan, publicado en The Scientist, Volume 19, Issue 1, Jun. 6, 2005. [Ver texto cvompleto más abajo]
Rercursos asociados
La evaluación por pares en cuestión: ¿mérito, amiguismo o resentimientos?, 2008
The British Academy, the national academy for the humanities and the social sciences Peer Review: The Challenges for the Humanities and Social Sciences, 2007
European Science Foundation, Peer review. Its present and future state
Arts and Humanities Research Council, Peer Review and the Evaluation of Digital Resources for the Arts and Humanities, 2006 [PDF]
Arts and Humanities Research Council and the Higher Education Funding Council for England, Use of research metrics in the arts and humanities, 2006 [PDF]
Paul Ginsparg, Can Peer Review be better Focused?, 2003
Thomas von Foerster, The Future of Peer Review, 2001
The British Academy, the national academy for the humanities and the social sciences Peer review
Wim D’Haeze, The Peer-Review Reviewed

How to Fix Peer Review
by David Kaplan, Ph.D.
Despite its importance as the ultimate gatekeeper of scientific publication and funding, peer review is known to engender bias, incompetence, excessive expense, ineffectiveness, and corruption. A surfeit of publications has documented the deficiencies of this system.[1-4] In September, the fifth in a series of international congresses concerned with how peer review can be improved will convene in Chicago. Yet so far, in spite of the teeth gnashing, nothing is being chewed.
Investigation of the peer-review system has failed to provide validation for its use.[1] In one study, previously published articles were altered to disguise their origin and resubmitted to the journals that had originally published the manuscripts.[5] Most of these altered papers were not recognized and were rejected on supposed “scientific grounds.” Other investigators found that agreement among reviewers about whether specific manuscripts should be published was no greater than would be expected by chance alone.[6]
Peer review subsumes two functions. First, peer reviewers attempt to improve manuscripts by offering constructive criticisms about concrete elements such as the application of a technique, the strength of results, or the cogency of an argument. The second function of peer review is to render a decision about the biological significance of the findings so that the manuscript can be prioritized for publication. I propose reforming peer review so that the two functions are independent.
Review of a manuscript would be solicited from colleagues by the authors. The first task of these reviewers would be to identify revisions that could be made to improve the manuscript. Second, the reviewers would be responsible for writing an evaluation of the revised work. This assessment would be mostly concerned with the significance of the findings, and the reviewers would sign it.
After receiving the final assessments from several different reviewers, the authors could decide to submit to a journal, sending the manuscript and the signed reviews together. The editors, carrying out the second function of peer review, would then decide to publish or not based solely on this material. The reviewers’ identities would be revealed in the publication.
I believe there would be several significant effects of this change in peer review. First, the authors would submit only positive assessments. Consequently, reviews would emphasize why a manuscript should be published instead of why it shouldn’t be. Second, investigators would be less likely to publish insignificant findings. They would have to ask colleagues to put their names on the manuscript; consequently, the tendency would be to ask for support for more complete and more compelling sets of findings.
Third, reviewers would be forced to account for their comments. They could not perform just a cursory look without the authors realizing the review was not insightful and did not represent an honest effort. Fourth, although it would be possible to have close friends and relatives review a manuscript, the editors would see who was supporting publication. In their deliberations, the editors would consider the breadth of the reviewers and their relationships to the authors and to the conceptualization promulgated in the manuscript.
Fifth, the editors would be free from adjudicating between authors and reviewers. They could concentrate on the specific arguments put forth for publication. Moreover, the process would be considerably streamlined, since there would be no need to send the manuscript out for review.
This revision of peer review would change the incentives for all involved. The authors would tend to publish results that represent more complete findings and be more satisfied with the outcome, because they could exert lots of control over the review process. The reviewers would tend to be more honest in their evaluations, not wanting to praise work they consider flawed, because their names would be attached to it. Reviewers would not give a cursory and willfully negative evaluation, because the authors could simply not forward their comments. It would be in the reviewers’ best interests to help improve manuscripts that have flaws but are potentially important.
The editors would emphasize publication of manuscripts that have the broadest support among scientists in the relevant community or that have the greatest potential to influence the community. Their jobs would be easier because the number of manuscripts submitted would be fewer, although of more substance. This tendency would be facilitated by editors’ publicizing the stringent acceptance requirements. For example, editors could request manuscripts with support from reviewers from the same institution and from other institutions. They could request reviewers in the same field and reviewers in related fields.
Peer review is broken. It needs to be overhauled, not just tinkered with. The incentives should be changed so that: authors are more satisfied and more likely to produce better work, the reviewing is more transparent and honest, and journals do not have to manage an unwieldy and corrupt system that produces disaffection and misses out on innovation.
References:
1. T Jefferson et al, “Measuring the quality of editorial peer review,” J Am Med Assoc 2002, 287: 2786-90. [Publisher Full Text]
2. PA Lawrence “The politics of publication,” Nature 2003, 422: 259-61. [PubMed Abstract][Publisher Full Text]
3. DF Horrobin “The philosophical basis of peer review and the suppression of innovation,” J Am Med Assoc 1990, 263: 1438-41. [Publisher Full Text]
4. M Enserink “Peer review and quality: A dubious connection?” Science 2001, 293: 2187-8. [PubMed Abstract][Publisher Full Text]
5. DP Peters, SJ Ceci “Peer-review practices of psychological journals: The fate of published articles, submitted again.” Behav Brain Sci 1982, 5: 187-96.
6. PM Rothwell, CN Martyn “Reproducibility of peer review in clinical neuroscience,” Brain 2000, 123: 1964-9. [PubMed Abstract][Publisher Full Text]
Reprinted with kind permission of The Scientist
Volume 19 |Issue 1 | Page 10 | Jun. 6, 2005