IEEE

When Peer Reviews Vanish: A Hidden Reviewer Decides Your Fate

After months of denial, the record now shows what authors suspected all along.

4 min readMachine Learning
When Peer Reviews Vanish: A Hidden Reviewer Decides Your Fate
[UPDATE - EIC confirmed ghost reviewer]How to get rejected by IEEE T-PAMI with 'Excellent' scores?[D]

The paper received three reviews that were, by any reasonable standard, excellent. The authors were told the decision came down to a fourth reviewer, one who had, according to the record, submitted a positive assessment. That review then vanished. It took six months of persistent effort, and an official investigation by the IEEE Computer Society Committee on Integrity, for the Editor-in-Chief to finally confirm what the authors already knew: the fourth review existed, and it had been favorable. The rejection, it turns out, was based on a ghost.

This is not a story about a close call or a subjective difference of opinion. It is a story about the machinery of academic publishing failing at its most basic level, and about what happens when the process designed to protect the integrity of science quietly erodes the trust of the people who fuel it. We spend a lot of time discussing how to build better models, how to train them faster, and how to make them more accessible. But the infrastructure that evaluates and disseminates that work is often treated as a static given, something to be navigated rather than questioned. This case is a stark reminder that the review process is a human system, and like any human system, it is vulnerable to error, bias, and, in this case, a kind of procedural obscurity that borders on the Kafkaesque.

For the researchers in our community who are building their careers on rigorous work, the takeaway here is not to become cynical, but to become precise. The Unlock LLM Training: A Practical Guide to Distributed Algorithms post we published recently highlights how much of our field's progress depends on understanding the underlying infrastructure. The same logic applies here. You cannot optimize what you cannot measure, and you cannot challenge a decision you cannot see. The authors' persistence in pursuing this matter, documenting every step, and forcing the committee to acknowledge the discrepancy, is a masterclass in operational diligence. It is the academic equivalent of checking your gradients, logging your losses, and verifying your tensor shapes. It is unglamorous, but it is essential.

The deeper issue, however, is one of accountability. The AE in this case made a decision based on a phantom review, and the subsequent correction, while validating, does little to address the six months of lost time, the delayed revisions, and the emotional tax of fighting a shadow. This connects to a broader pressure we see across academia, where the stakes of a single publication can feel as high as a Neurosurgery Match Requirements Highlight Growing Pressure on Medical Students. The system is straining under its own weight, and the people caught in the middle are the ones who are told to trust the process, even when the process appears to be gaslighting them.

What we would tell a reader who asked us about this is simple: document everything, and do not accept a "no" that does not have a clear, verifiable foundation. The specific detail to watch here is how IEEE handles the AE in question. A correction on the record is one thing, but a change in behavior is another. If the AE faces no consequences for this error, then the message is clear: the process is a formality, and your work is subject to the whims of a hidden layer. That is not a future we should be building toward. The authors were right to push, and they have given the rest of us a playbook for what to do when the ghost in the machine turns out to be a person. The question now is whether the machine will fix itself, or whether it will simply learn to hide its ghosts better.

From Machine Learning

Background : Our T-PAMI submission was rejected despite receiving three highly favorable reviews. The AE inadvertently revealed that the decision relied on negative comments attributed to a “fourth reviewer.” However, the actual fourth reviewer had submitted a positive review, which subsequently disappeared from the review record under the AE’s handling. We have spent the past six months pursuing this matter with IEEE. (Original post: [How to get rejected by IEEE T-PAMI with 'Excellent' scores?[D])

Read the original at Machine Learning