10.08.2023 Publications

Rethinking the article review process

Peter Schulz, a scholar of scientific communication at Unicamp, discusses the challenges of making the peer-review process more open and transparent

Ricardo Muniz

Manuscripts selected for publication in scientific journals are usually submitted to rigorous quality screening. The evaluation process is carried out by scientists who work in the same area of research as the authors, hence the name peer review. During this process, scientists with robust knowledge of a given subject are expected to make suggestions on how to improve the quality of the evaluated papers.

Peer review therefore has the potential to be an instrument for learning and improving manuscripts, but this is not what happens, according to physicist Peter Schulz, a professor at the School of Applied Sciences of the University of Campinas (UNICAMP) and a scholar of scientific communication.

“These days, peer review is limited to rejecting or accepting manuscripts for publication. The dialogue between reviewers and authors has been stifled, which is really bad news,” says Schulz, who was UNICAMP’s Communications Secretary between 2017 and 2021.

A former professor at UNICAMP’s Gleb Wataghin Institute of Physics, where he taught for two decades, Schulz now focuses on scientometrics, a field that analyzes quantitative aspects of science, and scientific dissemination initiatives, such as his column in Jornal da Unicamp.

In an interview with Science Arena, Schulz suggests that improvements in the peer-review process should draw inspiration from the open science movement, which seeks to provide greater transparency in scientific activity. “In practice, a more transparent review would mean opening reviewers’ opinions to the entire community. That way, the peer review would itself be scrutinized, allowing problems related to integrity to be identified.”

The researcher recognizes, however, that such a proposal faces resistance, especially from major scientific publishers.

Science Arena – What is the origin of the peer-review process?

Peter Schulz – Today’s scientific publication process, including peer review, is a construct that began with the institutionalization of modern science. The articles published in journals in the past, however, were completely different from today. There were many formats of articles published in journals considered scientific, such as the UK’s Annals of the Royal Society, as far back as the seventeenth century.

Editors were exclusively responsible for choices, selections, comments, and accepting or rejecting manuscripts. As the number of articles submitted for publication increased—and consequently the number of journals—a body of editors was established at the turn of the nineteenth century. Members of this editorial group began to ask for help and suggestions from fellow experts to help curate the manuscripts.

When did this process become a symbol of quality and legitimacy for scientific publications?

Peer review has consolidated itself as a true institution, to the extent that it is now an essential requirement if a scientific journal wishes to be indexed in international databases. This “maturation” occurred in the middle of the twentieth century. It is quite recent. Up until the 1940s, there were still journals that did not do a peer review, with the editor in chief remaining the one who decided what would be published.

It is important to ask what reasons led to the institutionalization of peer review. The main argument is that the process ensures the technical quality of published articles. In practice, however, it has become a mechanism for expanding scientific publication. Peer review made it possible to streamline the publishing process, consolidating the market for large commercial publishers known for their high profit margins. It is worth noting that peer review is mostly outsourced: most of the time, the reviewers work voluntarily.

How has the increased volume of publications affected quality assurance?

A good editor should review the peer reviews. This means checking whether the review was done with technical rigor, whether it was consistent, whether it actually explained why a particular manuscript should or should not be accepted. However, as the number of articles continues to increase, the volume of work is also growing. Decisions need to be made quickly and this affects the quality of the review process. So the final decision on the manuscripts to be published, despite in theory still belonging to the editor, is effectively made by the reviewers. The central question, therefore, is who is reviewing the peer review?

Could you give us an example?

At the beginning of my career, it was common for reviewers to give negative opinions. I then countered their review, defending my arguments, and they answered back, so there was a two-way dialogue. But this dynamic has become increasingly rare. I recently submitted a manuscript and one reviewer said the work was “useless,” while another found the work interesting but suggested some changes. I replied to the reviewers, but my comment never reached them. It ended up with the editor, who told me: “it is not our editorial policy to encourage discussions between those who submit papers and the reviewers.” This type of thing, which is increasingly common, does not help improve the quality of scientific literature.

In the past, reviewers played an important role in helping to improve the quality of a manuscript. Nowadays, the peer-review process has been increasingly limited to the ‘mechanical’ task of accepting or rejecting manuscripts, as just another step in the paper production line.

Peer review could play a more impotant role in the construction of scientific knowledge. It is no longer a learning tool for authors, and that is really bad news. The dialogue between reviewers and authors has been stifled.

Has it become a production line without a “seal of quality,” then?

The justifications put forward by researchers are along the lines of “I need to publish a lot to justify the funding I received.” This is ingrained, even among junior researchers at the beginning of their careers. I ask colleagues: “why do you have ten or more undergraduate or master’s students working on these articles?” The answer I usually get is that they need to do this to maintain a high level of publications, to justify the money they receive from funding agencies. This is a culture that has taken root and it has become a sort of vicious cycle.

For example, in some fields you find research that needs to be done, but the papers are actually nothing more than technical reports. These reports end up receiving the status of scientific articles, because they are published in prestigious journals, have undergone peer review, and are indexed in international databases.

However, many of them represent just a small part of the broader research. This phenomenon is known as “salami publication,” because it occurs when findings from a scientific study are “sliced” into several smaller results in order to be published in a series of papers. In these cases, the scientific article is an end in itself.

Is there any way to resolve this?

One possibility is open science, in the sense of making studies and research data public and accessible to all sectors of society and the scientific community. This can be done through repositories created by universities and preprint platforms, which share papers that have not yet been peer-reviewed.

A sharing culture in which your work is uploaded to an open science platform, subject to the scrutiny of other researchers who can comment, critique, and suggest citations, is perfectly viable. In fact, there are already many initiatives taking this approach, endeavoring to establish an open and transparent peer review.

This does not eliminate the conventional work of reviewers, it just expands the scope of the peer review, creating better conditions for a collaborative culture of knowledge construction. But putting initiatives of this type into practice takes work and inconveniences large publishers that still profit from publishing studies in closed or hybrid access systems.

* This article may be republished online under the CC-BY-NC-ND Creative Commons license.
The text must not be edited and the author(s) and source (Science Arena) must be credited.