The variance in ICML review scores is not a bug in the system; it is a direct reflection of how different batches are assembled and who gets assigned to them. When one reviewer reports that almost nothing in their pile clears a 3.5 while another sees a cluster of 3.75s, the simplest explanation is that they are not reviewing the same kind of papers, nor are they being judged by the same standards. Domain matters, but so does the arbitrary luck of which reviewers land on which submissions. That is uncomfortable to hear, but it is also the truth.
For you, the author, this means that a single score is not a verdict on your work's quality. It is a data point drawn from a noisy process. If your paper lands in a batch where reviewers are stricter, or where the topic is more contested, your score will reflect that context more than your contribution. The practical takeaway is to stop reading your individual scores as if they were precise measurements. Instead, read the reviews for the specific criticisms you can act on, and treat the numeric score as a rough signal that tells you how much revision effort to invest, not whether your idea has merit.
ICML does not fully account for this variance, despite what some might hope. There is no secret normalization that equalizes every batch before scores are released. What the conference does is rely on the sheer volume of reviewers and the discussion phase to smooth out the extremes, but that only works if you are willing to sit with the discomfort of a score that feels too low or too high. The system is not designed to give you a fair score; it is designed to produce a ranked list of papers that can be defended as reasonable, not perfect.
So what should you do with this knowledge? Stop comparing your batch to your neighbor's. Start asking reviewers for specific, actionable feedback during the discussion period, and use that to improve your paper regardless of the number attached. The variance will not disappear, but your ability to navigate it can. That is the only control you have, and it is worth more than any score.