Peer review, the assessment procedure of a scholarly manuscript carried out by external experts prior to publication, is an essential part of scholarly communications. It has recently been described as the cornerstone without which “the whole edifice of scientific research and publication would have no foundation”. (1) However crucial, peer review goes nonetheless mostly unrewarded.

Researchers are always struggling for time between conducting and documenting their research, obtaining funding through grant applications, and keeping apace with the literature in their field. A large proportion of researchers also have to deal with the tasks of teaching and mentoring students, managing labs, and travelling to present their findings. It seems paradoxical, therefore, that a fundamental yet time-consuming task such as peer review is not formally incentivized, especially in our times of budgetary restrictions for science, growing competition for grants, and increasing emphasis on productivity.

Prof. Philippe Baveye

Prof. Philippe Baveye

The reviewing crisis

For Prof. Philippe Baveye of the SIMBIOS Centre, Abertay University, this very real problem is nonetheless only the tip of the iceberg: “Now more than ever, many more manuscripts are submitted to journals than really deserve to be. A huge amount of them are junk, submitted for reasons other than the sharing of new knowledge, which understandably nobody wants to review. It is in this context that the peer-review crisis has to be interpreted.”

Prof. Bernard Grabot

Prof. Bernard Grabot

Although there have been ideas for penalising late reviewers (2) as an incentive for prompt reviews, the majority of suggestions focus on positive reinforcement. (3) Prof. Bernard Grabot, of the Ecole Nationale d'Ingénieurs de Tarbes, France, agrees that this is the right approach: “In my opinion, the idea is to encourage people to review; we should therefore avoid any penalty, even for ‘poor’ reviewers, as people would prefer not to respond than risk a bad evaluation.”

Peer-review metrics

While some journals do provide access to e-content or Abstracting & Indexing services such as Scopus, publish lists of reviewers and/or frequent reviewers, or even pay reviewers a token sum for each completed review, most peer reviewing goes unrewarded. The most recent proposals to change this have advocated the application of scientometrics to peer review. (4)

Dr Elena Paoletti

Dr Elena Paoletti

In November 2009, Dr Elena Paoletti of the National Council of Research, Italy, proposed the Reviewer Factor: a simple indicator based on the number of reviews multiplied by the citation influence of the journal, which would be “a concrete way to provide public recognition of [reviewers’] attitude to evaluation and importance in the field, and a succinct measure of [their] experience in peer review.” (5) Late reviews may or may not be taken into account.

Dr Pedro Cintas

Dr Pedro Cintas

Meanwhile, Dr Pedro Cintas of the University of Extremadura, Spain, suggested a Peer Review Index: a metric or “peer review capability [which] would be the quotient between the number of papers evaluated (q) and the number of papers published (p) within a given period.” (6) This could be made to incorporate the quality of the reviews in terms of relevance and usefulness, as evaluated by the editors.

Prof. Bernard Grabot comments: “Concerning what would make a ‘good’ index, the discussion is open […] The important thing would be – if possible – to get a single index for a reviewer, summarising his/her activities for most of the journals [...] but I suppose it is quite difficult. It would be useful to get similar indices for all the journals, which could then be computed at reviewer level.”

While Prof. Philippe Baveye does not deny the usefulness of these types of indicators, he believes that they are only part of the solution: “Certainly, peer-reviewing effectiveness indices like those that are being proposed would help, […] but that would not be enough. The solution to the problem has to be sought by attacking the ‘publish or perish’ mentality directly, wherever it manifests, and by reducing drastically the number of articles published in most disciplines.” (7)

Although there is a clear need for the academic community to incentivize peer review in order to preserve a fast and efficient quality check of scientific manuscripts submitted for publication, there is as yet no uniformly established method to do so. With the recent incorporation of the nascent reviewer metrics, the issue has the potential to turn into a hotly debated topic.

Useful links

Rewarding reviewers – could a Reviewer Factor be a solution?
Increasing visibility and recognition of reviewers – is a Peer Review Index a possible solution?
Sticker shock and looming tsunami: the high cost of academic serials in perspective

References:

(1) (2008) “The pitfalls and rewards of peer review”, The Lancet, vol. 371, issue 9611, p. 447.
(2) Hauser, M & Fehr, E (2007) “An incentive solution to the peer review problem”, PLoS Biology, vol. 5, issue 4, p. e107.
(3) Baveye, PC, Charlet, L, Georgakakos, K P, & Syme, G (2009) “Ensuring that reviewers’ time and effort are used efficiently”, Journal of Hydrology, vol. 365, issue 1–2, pp. 1–3.
(4) Baveye, PC & Trevors, JT (2010) “How Can We Encourage Peer-Reviewing”, Water, Air, & Soil Pollution, pp. 1-3, article in press DOI 10.1007/s11270-010-0355-7.
(5) Paoletti, E (2010) “Rewarding reviewers – could a Reviewer Factor be a solution?”, Elsevier Reviewers’ Update, issue 4, March 2010.
(6) Cintas, P (2010) “Increasing visibility and recognition of reviewers – is a Peer Review Index a possible solution?”, Elsevier Reviewers’ Update, issue 4, March 2010.
(7) Baveye, PC (2010) “Sticker shock and looming tsunami”, Journal of Scholarly Publishing, vol. 41, issue 2, pp. 191-215.
VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)