The peer review system for AI conferences is showing cracks that can no longer be ignored, and the situation described here, a review flagged as 100% AI-generated with confidence 1, contradicting two higher-confidence reviews, is the kind of signal that demands direct action. What we are seeing is not an isolated glitch but a symptom of deeper pressures that have been building for years, as we explored in Why ICLR authors and reviewers are going silent on rebuttals and How Reviewers Confidence Shapes Split Decisions in AI Research. The practical question for every author facing this is not whether to flag it, but how to do so without undermining their own credibility.
Our view is straightforward: send the confidential comment to the Area Chair. The reviewer's confidence of 1 is already an admission that they are guessing, and when combined with a detector score and the telltale pattern of weaknesses buried in the appendix, it becomes a pattern rather than a coincidence. Area Chairs are not looking for proof beyond a reasonable doubt, they are looking for enough signal to justify a deeper look. A detector score alone is not actionable, but a detector score paired with specific content issues (generic criticisms, misplaced focus on minor appendix details, and a confidence level that screams uncertainty) gives them the basis to discount that review or, at minimum, to weigh it less heavily in the meta review. The real risk is not that you come across as defensive, but that you stay silent and let a low-quality review shape the outcome of months of work.
What weight does a confidence 1 review realistically carry? In practice, very little on its own. A single low-confidence score is often treated as noise, especially when the other reviewers are aligned and confident. But the problem is that noise can tip a borderline decision. When scores are split 3, 3, and 2, that 2 becomes the deciding factor in a reject-or-accept threshold. So the question becomes: are you willing to let a reviewer who does not even trust their own judgment decide your paper's fate? We think not. The community needs to develop clearer norms here, and authors have a role in shaping them by flagging these cases professionally.
The open question that remains is what happens next. Have Area Chairs seen enough of these AI-generated reviews to develop a standard response? Based on the experiences shared, the answer is still no, most are navigating this case by case. That means authors who encounter this should document everything: the detector output, the specific weaknesses cited, the confidence scores, and their own rebuttal responses. This is not about policing reviewers, but about protecting the integrity of a process that already struggles with scale, as noted in Exploring a hard cap on ML conference papers to sharpen research focus. The concrete takeaway is this: if you see a review that looks like someone asked an AI to skim your appendix and produce weaknesses, say something. The system cannot fix what it does not see.