A study of 124 rapidPeer submissions found that Claude Opus 4.8’s tuned reviews were judged better than human-written reviews in 132 of 134 comparisons; two were undecided. Gary King and Steven Worthington’s paper measures review quality under rapidPeer’s rubric, not whether AI improves papers or should decide what gets published. Their paper is dated October 28, 2026.
A higher review score can show that feedback performed well under a rubric; it cannot, by itself, show that a paper improved or that an editor should accept it.
What the rapidPeer study compared
The researchers analyzed 124 scholarly-paper submissions and 134 human reviews from 2019 through 2026. Ten submissions had two independent human reviews. The researchers generated AI reviews for the same submissions using Claude Opus 4.8 at its maximum reasoning setting.
They tested two setups. The “matched” reviews used the manuscript and rapidPeer guidelines without external tools. The “tuned” reviews used a purpose-built prompt and web retrieval. Each AI review was generated in a separate call.
GPT-5.5, a model from a different family, judged the reviews against rapidPeer’s criteria. The rubric covered sources, weaknesses, methods, theory and ideas, framing, communication, and figures and tables. In the tuned condition, the paper reports higher mean AI scores in all seven areas.
The AI judge and the editors assessed different samples
For the tuned reviews, the paper’s detailed results report 132 AI wins, no human wins, and two undecided outcomes across 134 comparisons in the reviews-only assessment by GPT-5.5. The manuscript-access comparison had the same outcome counts.
Three former journal editors also assessed 10 randomly selected, blinded pairs. They did not read the manuscripts. Across their 30 individual judgments, they favored the AI review 26 times. That 26-of-30 result describes the editors’ votes on those 10 pairs, not the outcome across all 134 comparisons.
The audit still found errors in AI reviews
A blinded, model-based audit flagged 57 of 1,039 AI claim assessments (5.5%) and 152 of 956 human claim assessments (15.9%) as incorrect relative to the manuscripts. These percentages apply to claim assessments, not entire reviews. The audit also flagged at least one incorrect claim in 46 of the 124 AI reviews.
The paper says a human check of 90 claims agreed with the audit on 87. That check adds context to the model-based assessment, while the review-level count shows that flagged errors appeared in more than a third of the AI reviews.
What the findings say about peer review
The study assessed constructive feedback to authors. It did not measure whether authors improved their papers after receiving AI feedback, or whether AI led to better journal publication decisions. The editors in the 10-pair sample did not examine the manuscripts, either.
King and Worthington disclose that they conceived and operate rapidPeer, the service that supplied the study corpus. They also say the research received university IRB approval and that enterprise AI agreements prevented providers from training on user data. Inside Higher Ed reported on November 5, 2026, that the paper had not yet been peer reviewed.
