From Polarized Reviews to a Borderline Accept

Navigating the complexities of ICML reviews can be challenging, especially when facing polarized feedback.

3 min readMachine Learning

The math here is straightforward: three reviewers think your paper is a solid accept, one thinks it is a rejection, and the average lands at 4.25. That is not a coin flip. That is a borderline case where the outcome hinges less on the raw numbers and more on how the area chair reads the confidence behind them. And in this instance, the confidence is doing a lot of work, on both sides.

The two reviewers who moved from 4 to 5 did so with clear, written justifications. They engaged with your rebuttal, resolved their concerns, and explicitly stated they believed the final version would be strong. That is not a passive endorsement. That is an active, reasoned vote in your favor. Meanwhile, the single dissenting reviewer raised their confidence from 4 to 5 while keeping the score at 2, dropped originality to 1 despite every other reviewer giving a 4, and became more aggressive after you responded constructively. That pattern suggests a reviewer who came in with a fixed position, not one who was genuinely weighing the merits. Area chairs see this dynamic regularly. They are trained to weigh reviewer intent, not just the final number.

So what does this mean for you practically? First, do not treat the negative reviewer as the voice of truth. Their critiques may contain a kernel of validity, most do, but the escalation in tone and the refusal to engage with your responses signals a lack of intellectual flexibility. That is not a reflection on your paper. Second, understand that the AC's job is to synthesize signals, not average them. A 5, 5, 5, 2 with confidence scores of 4, 3, 4, 5 is a different signal than a uniform 4, 4, 4, 4. The former says "strong consensus with one outlier"; the latter says "everyone is lukewarm." You have the former. That works in your favor.

The uncomfortable truth is that borderline papers often come down to the AC's risk tolerance and the pool of other papers in the same batch. If the AC is confident in their own judgment, they may override the outlier. If they are cautious, they may default to the negative review as a tiebreaker. You cannot control that. What you can control is how you frame the situation in your own mind: you did the work, you responded professionally, and you moved two reviewers from "meh" to "strong accept." That is a win, regardless of the final decision. If this paper does not make it, it will not be because you failed to persuade. It will be because one reviewer chose to be a wall, not a door. Keep that perspective, and keep submitting.

From Machine Learning

Hi, rebuttals recently finished, and I wanted to share my paper's scores to ask for thoughts on this, and whether this situation is borderline and dependent on the AC.

My paper started out as 5 (4), 4 (4), 4 (3), 2 (4). After the rebuttals, I ended up at 5 (4), 5 (4), 5 (3), 2 (5). This makes the average 4.25, with a confidence average of 4.

Read the original at Machine Learning