Blind review falters when a quick search reveals the authors.

In the peer review process, the identity of authors can inadvertently shape our evaluations, even if we strive for objectivity.

3 min readMachine Learning

Blind review has always been a fragile experiment in human nature, not just a protocol. When a first-time reviewer notices that the two papers they ranked highest are also the only ones already on arXiv, the system's mask slips. It is no longer a question of whether bias exists, but how often it quietly shapes decisions without anyone logging it. The reviewer's own admission, that they tried to work out if revealing the author's identity influenced their judgment, is the most honest sentence in this entire discussion. It also reveals the core problem: we cannot unsee what we have seen, and pretending otherwise is naive.

The practical takeaway here is uncomfortable but straightforward. If you are a reviewer, your curiosity is not the enemy; the lack of safeguards around it is. The moment a paper is searchable with author names attached, your evaluation is no longer purely about the work. It becomes a mix of merit and reputation, however subtle. That does not mean every reviewer is corrupt or careless, but it does mean the process has a leak. For authors, this creates an uneven playing field. Those who post early on arXiv, often from well-known labs, gain an unearned advantage. Their work gets a halo effect before a single reviewer formally opens the file. Meanwhile, a rigorous but anonymous submission from a less visible group starts a step behind, through no fault of its own.

What makes this story particularly telling is that the reviewer caught themselves. They did not set out to game the system. They got bored, searched, and then had to live with the cognitive dissonance. That is the quiet tragedy of blind review in the digital age. The tools we use to share knowledge have outpaced the rules we use to evaluate it. A quick search is not a moral failing, but it is a crack in the foundation. And once that crack exists, it does not matter if most people stay honest. The perception of bias is enough to erode trust, and trust is the only real currency a review process has.

The fix is not to ban Googling, because you cannot police curiosity. Instead, we should stop pretending that double-blind review is truly blind when the internet makes it trivially easy to pierce. Journals and conferences need to acknowledge this reality and either embrace open review, where names are known and biases are addressed head-on, or enforce stricter submission windows that keep papers off public servers until decisions are made. The reviewer's experience is a signal, not an anomaly. Listen to it. If we want evaluation to be fair, we have to design for the way people actually behave, not the way we wish they would. That means building systems that assume the search bar exists, and then asking what integrity looks like from there.

From Machine Learning

Let's be honest, at some stage of the review process. A lot of us have gotten bored and tried to Google the papers we are reviewing. And sometimes those papers might have already been uploaded onto arXiv with the identity of the authors. Which we then tried to look them up.

As a first-time reviewer, I noticed the top 2 papers in my batch happened to be the only papers in my batch that is on arXiv. I am trying to work out if revealing the author's identity had influenced my decision. Or it's just a coincidence.

Read the original at Machine Learning