50 shades of review reports
What is the purpose of a review report?
Reading review reports is part of an editor’s daily routine. Their content, however, is not at all standard. I am not talking about the scientific content, of course; every study is different and as such will receive distinct feedback. What baffles me is how different the reports are in length, thoroughness and tone of voice. Some reports are no more than a paragraph, others several pages long. Some consist of a couple of statements on what is missing, others contain concrete experimental ideas and requests. Some are friendly and encouraging, others straight-out offensive.
Let me share with you how I feel about sending papers out to review: I have a study which – from a literature search – seems to contain some exciting new findings, I contact expert scientists working in the relevant field and say “I think you and your peers may like this”, and then I wait for them to tell me whether or not I was right and what the authors can do to improve their study and solidify their conclusions. In other words, to me, a review report is a generous piece of advice not only for me as an editor but also for the authors as experimenters.
For me as an editor, a review report is first and foremost a generous gift, gift of time and effort of the reviewer. I cannot thank our referees enough for generously sharing their opinion and thoughts on the papers with me. The report of my dreams informs me about the timeliness of the topic, the novelty of the results, and the technical rigour of the results. It asks questions that remain unanswered, challenges the conclusions and gives the authors concrete instructions about what is missing to make their study a rock-solid piece of science.
Yet, different people seem to have a different view on what a review report is and the information it should transfer. Occasionally, a report is basically a well-written summary of the study at hand. Those reports demonstrate that the referee read the study, yet they do not contain the advice I seek. The content of the study was exactly how far my assessment went. Even though we editors judge papers related to our expertise, we cannot judge the technical aspects of the studies and thus require experts to help us. Any advice on how to improve the quality of a paper is appreciated.
Contrary to the previously described reports some reports make me think back to a former (lab) supervisor of mine, who once fondly chuckled: “It is incredible what one can suggest in a review report…” Lists of additional experiments with completely different setups are indisputably rich sources of inspiration and – in the best case – meant to help the authors. However, sometimes they can just appear as a recipe to shave a few years off some PhD student’s life.
I will cut this here and leave you with this: Everyone seems to have a slightly different view on what a review report should achieve. Acknowledge, criticize, inspire, challenge (…) – ideally, a review report hits all of these marks. I could continue for much longer, talking about different versions of review reports. But I will spare you and rather ask: What do you think a review report should achieve and on which aspect do you put your focus? Let me know in the comments below, I am genuinely curious. And if you have ever reviewed for Nature Communications, please let me reiterate: Thank you very much for being so generous with your time and knowledge; every minute you have spent on composing your review report is much appreciated.