[Explained] Boon, bias or bane? The potential influence of reviewer recommendations on editorial decision-making

[

Tennant, J. P., Penders, B., Ross-Hellauer, T., Marušić, A., Squazzoni, F., Mackay, A. W., . . . , Graziotin, D., Nicholas, D. (2019). Boon, bias or bane? The potential influence of reviewer recommendations on editorial decision-making. European Science Editing, 45(1), 2-4. doi:10.20316/ese.2019.45.18013

Open access. Click the link and read the paper.

Research studies usually end up with a report, called paper, which is submitted to a venue for publication consideration. Most fields submit their papers to academic journals (with editors), whereas we computer scientists also enjoy conferences with formal published proceedings.  In both cases, venues usually provide a technical evaluation of the submission in a process called peer review. The process is supervised by a person (editor in a journal) or a committee (program committee for a conference) and reviewers provide a formal recommendation (e.g., “accept with minor revisions” and “reject”) in addition to their report on the submission.

I was taken by surprise to see that not all academic fields explicitly ask reviewers for a formal recommendation. Their duty, when a recommendation is not asked, is to only comment on the merits and issues of a submission. The editor alone should take care of the submission fate, after reading the reports, and produce a report on their own with their editorial decision.

I united with a group of other editors  from many disciplines and co-authored an editorial (which was oddly also peer reviewed) for the journal of the European Association of Science Editors. We formulated several questions, for example, Should journals invite reviewer recommendations, either built into manuscript handling systems or within the reviewer reports? and Should authors see these recommendations? and What is the impact of these recommendations on editorial decisions?

We discuss pros and cons of these (and other) questions and hypothesize on how the process might change in the future. For example, asking reviewers to provide an editorial decision might help in distributing the decision workload and responsibilities, thus making the process more democratic. On the other hand, having recommendations by reviewers brings in the risk to “mathematicize” the decision process, as reviewer recommendations are numerical values in the end.

This post is part of the scicomm series [Explained]. Read about it.

About the author

dgraziotin

Dr. Daniel Graziotin is a senior researcher (Akademischer Rat) at the University of Stuttgart, Germany. His research interests include human, behavioral, and psychological aspects of empirical software engineering, studies of science, and open science. He is associate editor at the Journal of Open Research Software and academic editor at the Research Ideas and Outcomes (RIO) journal. Daniel was awarded an Alexander von Humboldt Fellowship for postdoctoral researchers in 2017, the European Design Award (bronze) in 2016, and the Data Journalism Award in 2015. He received his Ph.D. in computer science at the Free University of Bozen-Bolzano, Italy.

Add comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

About Author

dgraziotin

Dr. Daniel Graziotin is a senior researcher (Akademischer Rat) at the University of Stuttgart, Germany. His research interests include human, behavioral, and psychological aspects of empirical software engineering, studies of science, and open science. He is associate editor at the Journal of Open Research Software and academic editor at the Research Ideas and Outcomes (RIO) journal. Daniel was awarded an Alexander von Humboldt Fellowship for postdoctoral researchers in 2017, the European Design Award (bronze) in 2016, and the Data Journalism Award in 2015. He received his Ph.D. in computer science at the Free University of Bozen-Bolzano, Italy.