OpenReview

When feedback vanishes, trust in transparent review systems is tested

Transparency in peer review is fragile when comments vanish without explanation.

3 min readMachine Learning

The moment a comment disappears on OpenReview, trust gets tangled in the machinery of peer review. One author, posting as Terrible-Chicken-426, described how an Area Chair's summary of reviewer questions vanished, along with their own detailed reply. The AC posted on day one, the authors answered every point, and then both were gone. The author's suspicion is direct: did the AC delete the thread to avoid having an unjustified rejection look sloppy later? We cannot know the AC's intent, but the anxiety is rational. This is not a technical glitch story; it is a transparency story hiding inside a system that runs on trust.

What makes this worth pausing over is the asymmetry of power in academic publishing. The authors did the work of engaging with every weakness the reviewers raised. Their reply was a map of good faith. When that map disappears, the record of their diligence disappears with it. In a field where Unlock LLM Training: A Practical Guide to Distributed Algorithms shows how much complexity sits behind modern AI systems, the review process itself is another complex system. And like any distributed system, it has failure modes. Losing a comment thread is a failure of persistence. But the deeper issue is that OpenReview gives no audit log for authors to see who deleted what, or why. If you are a researcher, this is not an abstract annoyance. It is your professional narrative being edited without your consent.

Practical advice for anyone in this situation: screenshot everything, immediately. The moment you post a reply, capture the full thread. Do not assume the platform will preserve it. This is not paranoia; it is the same instinct that makes you keep backups of your training runs. The related story on Neurosurgery Match Requirements Highlight Growing Pressure on Medical Students shows how institutional pressure warps individual behavior. In peer review, the pressure is different but the dynamic is similar. Authors are expected to respond to criticism, but they are given no guarantee that their response will survive. That is a structural flaw. If OpenReview wants to be the trusted layer for AI research, it needs version control for comments, not just for papers.

The concrete detail to watch here is whether this author receives a decision that references the missing AC comment. If the paper is rejected and the decision cites points from that vanished summary, the deletion becomes evidence of a broken process. If the paper is accepted, the disappearance is an unfortunate accident. Either way, the fix is not to trust the platform more; it is to make the platform less opaque. Ask your AC for a record of the thread. Email the program chairs. And if you are a reviewer or an AC yourself, remember that your words have a half-life. Publish them like they might become the only trace of a hard conversation. That is the standard we should hold ourselves to, because right now, the only thing protecting an author's work is a screenshot they took before the page went blank.

From Machine Learning

Hi everyone, we noticed that the AC's comment, along with our reply, has disappeared, and we are wondering if anyone else has experienced the same thing.

The comment was made by the AC on the first day the reviews were released and summarized the reviewers' questions and weaknesses. We addressed all of their questions in our reply, but now both posts (the AC's comment and our response) are gone.

Read the original at Machine Learning