AI written reviews

How AI Is Reshaping the Peer Review Process for Better or Worse

Half the reviews for AAAI and ICLR 2027 look AI-generated.

3 min readMachine Learning

The peer review process is in critical condition, and we need to stop pretending otherwise. A veteran reviewer for AAAI 2027 and ICLR 2027 has laid out a damning case: roughly half the reviews they now see appear to be AI-generated, while papers themselves are increasingly polished by language models in ways that obscure genuine human contribution. This isn't a future problem, it is happening now, inside the very conferences that are supposed to certify rigorous science. We have already seen conferences like ICLR 2027 continue desk reject policies to filter low-quality AI submissions, but that approach only catches the most obvious offenders at the submission gate. The deeper rot is inside the review process itself, where the tools meant to accelerate discovery are quietly corroding the trust that holds academic validation together.

The signs the reviewer describes are chillingly specific: reviews with identical paragraph lengths, suspiciously uniform structure, and fields like "questions for authors" that were clearly copied from a changed form template. Smart quotes, Unicode dashes, and extra DOI links that no time-pressed human would bother adding. On the author side, papers now feature marketing-like headings and synonym-stuffed sentences that lose the thread of an argument. Only one out of three ICLR submissions felt entirely human-written. This is not a matter of style preferences, it is a systemic failure of the gatekeeping function. The community has been discussing detection tools, as our coverage of Examining AI Detection Tools and Their Role in Scholarly Review shows, but detection is a cat-and-mouse game that will never be won purely through software.

The reviewer's proposed fix is stark: disallow AI usage entirely, even for polishing tone and style. At first glance, that sounds extreme. But consider the logic. Before powerful LLMs, reviewers judged papers on substance, not prose polish. Allowing AI "polish" creates an expectation that everyone must use it to compete, which then forces reviewers to read through uniformly slick language that masks whether the thinking behind it is original. The practical consequence is that conferences are now reviewing AI-generated critiques of AI-polished papers, with human judgment squeezed into a vanishingly small role. This is not sustainable. We have published a guide on Keep Your Local AI Agent on Track with Practical Guardrails, and that same principle applies here: if you cannot enforce a clear boundary, the tool will drift into every crack.

The specific detail to watch is the honor system. The reviewer acknowledges that a total ban would rely on good faith, but argues that is preferable to the current ambiguity, where "you can use AI to polish" becomes an implicit mandate. That is a concrete choice facing every program chair and editorial board right now: either draw a bright line that says no AI in review text and no AI polish in submissions, or accept that peer review becomes a performance of competence rather than an actual test of it. The signal is clear, the system is already broken, and waiting for a technical fix that never arrives is not a strategy.

From Machine Learning

TL;DR: AI written reviews, and AI-writing in papers

I was going to give this a title "An Obituary of the Review Process", but in the interest of not sounding too defeatist, I will just say "Critical Condition" because of AI writing, both on the author side and the reviewer side.

Read the original at Machine Learning