Final Scores Shift: Why Reviewer Updates Demand a Closer

Final Scores Shift: Why Reviewer Updates Demand a Closer Look

Navigating the review process for ICML 2026 can be challenging, especially when scores fluctuate unexpectedly.

April 16, 20263 min readMachine Learning

The reviewer's score flip is not a mystery to be solved; it is a signal to be read. When a reviewer moves from 4 to 5 during rebuttal, then back to 4 after the AC discussion, they are telling you something specific: your response addressed their stated concerns, but the final conversation introduced a doubt they had not fully weighed before. That is not early rejection. That is a reviewer who is still deliberating, and the shift reflects a genuine reassessment rather than a closed door.

What this means for you is practical, not fatal. Your average dropped from 4.0 to 3.75, which is uncomfortable but not disqualifying. Many accepted papers land in that range, especially when the AC reads the reviewer's final justification as a nuanced position rather than a hard no. The reviewer's acknowledgment during rebuttal still stands as a record of your responsiveness. The later reduction suggests they are weighing your work against a competing concern, perhaps scope, perhaps novelty, perhaps a point they raised in the AC discussion that you never saw. That is not something you can control now, but it is something you can prepare for in the next round.

The real takeaway is this: do not treat the open review as a verdict. Treat it as a map of where the reviewer's mind is still unsettled. If the paper is rejected, that 4 with a written justification becomes your clearest revision guide. If it is accepted, the dip will not matter. Either way, you have gained a precise account of which concern survives rebuttal, and that is more useful than a clean score. You know what to address next time, and that is a position of strength, not weakness.

So check the review once more for the final justification, note the specific language they used when reducing the score, and use that as your anchor. Do not email the reviewer or the AC asking for an explanation; that rarely changes outcomes and can read as pressure. Instead, spend your energy drafting a response to that unresolved point as if you already have the chance to resubmit. That is the only move that improves your odds. A score shift is not a death sentence; it is a direction. Follow it.

From Machine Learning

one of my reviewers initially gave 4(3). addressed his concerns during the rebuttal. He acknowledged it and increased the score to 5(3) with final justification as well. checked open review randomly now, I can see he reduced it back to 4. am guessing he did this during the AC reviewer discussion? is this a sign of early rejection? My average was 4, which has now reduced to 3.75. do I still have any chance? Any comments would be appreciated.

When Academic Review Incentives Mismatch Research Progress

My 4444 (4443 pre-rebuttal) got rejected (as expected). Just copying a reply I wrote a couple of days ago before decisions were out: There seems to be a misalignment in the incentives of this year’s ICML reviews. The rebuttal phase is pushing hard to encourage reviewers to reconsider their scores, which has a good motivation. But in practice, it creates a distorted dynamic. ACs are seeking homogeneous ratings among reviewers. As a reviewer, I feel the pressure to increase my score to avoid prolonged back-and-forth discussions. I would assume there may be many reviewers who are not engaged but raise their scores just to end the discussion. At the same time, reviewers who are initially positive often seem reluctant to update their scores, even after their concerns are addressed. I came across a review that said: “Thank you for the rebuttal. The paper is valuable. The rebuttal addressed all my concerns.” (rephrased to avoid directly locating the paper) Yet the score remained at 4. It now makes me nervous (NOW I KNOW I WAS RIGHT!) since scores are inflated while the conference has a limited capacity. In a few days, we may see MANY uniformly positively rated papers rejected, just like last NeurIPS. I would prefer to roll back to how peer review originally was: reviewers provide honest and independent evaluations; AC assess their quality and consistency; and borderline cases are resolved through AC discussion. The current mechanism feels unnecessarily complex and makes the already bad situation worse. submitted by /u/AffectionateLife5693 [link] [comments]