The recent uproar surrounding the ICML acceptance rates sheds light on a growing concern within the machine learning community: the increasingly stringent review process that may inadvertently stifle innovation. With only about 6.5K papers accepted out of nearly 24K submissions, the rejected works do not necessarily reflect poor quality. Instead, they often represent valuable insights that simply did not fit the narrow criteria set by the reviewers. This trend echoes sentiments expressed in discussions about the conference system, such as in the post "Seems ICML is rejecting MANY unanimous positively rated papers," where researchers found themselves frustrated by rejections of papers that had garnered positive reviews.
The challenge lies not only in the acceptance rates but also in the nature of the reviews themselves. Many researchers report experiences of receiving feedback that feels disconnected from the core contributions of their papers. Comments like "Only 200 benchmarks included" or "I don't think this paper is 'novel'" seem to arise from a shallow understanding of the work's intent and significance. As noted in another relevant piece, "[D] Many times I feel additional experiments during the rebuttal make my paper worse](post/d-many-times-i-feel-additional-experiments-during-the-rebutt-cmn9khjv60e0zcf936i46x3e4)," the pressure to justify scores and respond to critiques can lead to a cycle of unnecessary revisions that dilute the original vision of the research. This situation raises significant questions about the effectiveness of the current review process, which appears to favor quantity of benchmarks over quality of insight.
Moreover, the cascading effect of rejected papers flooding other conferences, like NeurIPS, only exacerbates the problem. As submissions rise and acceptance rates plummet, we find ourselves in a cycle where the influx of papers overwhelms the review system, leading to a diminishing return on the feedback provided. Researchers are left to navigate an environment where acceptance seems arbitrary, and rejection can feel more like a reflection of the system's inefficiencies rather than the merit of the work itself. This raises a critical issue: if the purpose of publishing is to advance understanding of long-standing problems, how can we ensure that the review process truly serves this goal?
As we look toward the future of conference publishing and peer review, it is essential to consider what changes can be made to foster a more productive and supportive environment. Could we shift towards a model that prioritizes constructive feedback over mere gatekeeping? How can we ensure that the voices of new and diverse researchers are heard amid the noise of established norms? These questions merit careful consideration as the community grapples with the evolving landscape of academic publishing.
In conclusion, as we approach the next round of submissions for significant conferences such as NeurIPS, it is crucial to advocate for a review system that emphasizes understanding and collaboration over competition. The current state of conference publishing feels unproductive, and if we are to foster genuine innovation, we must rethink our approach to evaluation and feedback. The future of research depends on it, and it is a conversation worth having.