The theme of Peer Review Week (PRW) this year is ‘Rethinking Peer Review in the AI Era’. With increasing accessibility of powerful LLMs (Large Language Models) and the proliferation of AI-powered software in the scholarly communication ecosystem, this years’ theme prompts an important reflection on accountability when it comes to the use of AI in peer review.

This year at Karger Publishers for Peer Review Week, we want to explore our responsibility to reinforce appropriate use of AI in the peer review process, while balancing the opportunity to explore new technologies that could support editors and reviewers in ensuring the integrity of our publications. We close Peer Review Week this year by looking to the future with Prof. Daniel Acuña, a computer science researcher and the creator of an AI reviewer, ReviewerZero.

The challenge in peer review itself is clear; empirical data and editors’ experiences agree that finding suitable reviewers who are available to review is a growing difficulty. This is a challenge that can be solved in part with the use of technology, according to Prof. Acuña, “AI can be a great complementary tool by providing a first review, a first check of a manuscript. It can also check more holistically by checking multiple fields; it doesn’t get tired.”.

This isn’t without its problems. As Prof. Acuña explains, “if you don’t use AI that is meant for peer review, the AI tends to be very superficial, agreeing with and repeating whatever the reviewer writes.”. Of course, there is the considerable issue of bias in algorithms, which can reinforce the biases of its human counterparts. Recent research in the Acuña lab found that AI can have biases towards prominent institutions, prominent names and certain regions of the world. As an industry, he cautions that we need to be aware of these biases and work to remove them.

There remains reason for optimism, AI can be a powerful companion for Editors, for the benefit of the integrity of the publication. Many new tools are becoming available to Editors to identify potential research integrity concerns. LLMs may also support Editors in the review of the content, “LLMs have the ability to capture aspects that are holistic, for example, a method doesn’t match the figure, checks that were very time-consuming to do before.”, Prof. Acuña shared. By providing Editors and Editorial teams with information to inform their review and integrity checks, AI can enhance the strength of the peer review process.

AI will not replace the crucial role of human experts in peer review. However, it is clear that there are opportunities and an appetite for embracing technology that can benefit and support researchers, reviewers, Editors and editorial offices in the publication of robust, trustworthy articles.