Why doesn't the ML research community limit the number of submissions per author? [D]

The frustration voiced by /u/alafaya101 regarding the volume of submissions in the machine learning research community resonates deeply, particularly given the recent challenges with the Association for Computational Linguistics (ACL) Area Review Rounds (ARR). The sheer number of papers flooding venues like NeurIPS and ICML is demonstrably impacting the quality of reviews, a concern echoed in discussions around withdrawing from ACL ARR and resubmitting to a workshop [Withdraw from ACL ARR and resubmit to a workshop?]. This isn't merely an inconvenience; it speaks to a systemic issue threatening the integrity of peer review and, ultimately, the advancement of reliable AI research. It’s a stark contrast to established practices in fields like Security (CCS) and Computer Architecture (DAC), where limits on submissions per author are commonplace and contribute to a more manageable workload for reviewers. The question then becomes: why this divergence, and what cultural factors are at play within the ML community?

One potential explanation lies in the relatively recent explosion of interest and investment in machine learning. The field's rapid growth has attracted a massive influx of researchers, many with limited experience navigating established academic norms. This influx, coupled with a sometimes-aggressive publishing culture, has created a pressure to maximize output, leading to authors submitting multiple papers simultaneously. Furthermore, the democratization of AI tooling—making it easier than ever to conduct experiments and generate results—further accelerates the production of research, exacerbating the problem. The focus on novelty, often prioritized over rigorous evaluation, can also lead to a proliferation of incremental contributions that, while perhaps interesting, don't warrant the extensive review effort they demand. Consider, for instance, the efforts to predict human preference in projects like imagebench.ai [Predicting human preference for generated image pairs using HPSv3], a task that often generates numerous variations and experiments, all potentially warranting publication.

The consequences of this imbalance extend beyond reviewer burnout. A rushed review process inevitably leads to superficial evaluations, increasing the likelihood of flawed or poorly substantiated research entering the literature. This, in turn, can lead to a proliferation of "noise" – papers that lack significant impact or reproducibility – hindering genuine breakthroughs and making it harder for researchers to discern valuable contributions. While some might argue that a free-for-all submission system encourages innovation by allowing researchers to explore a wider range of ideas, the current situation suggests the opposite: a deluge of submissions can stifle innovation by overwhelming the review process and diminishing the quality of feedback. The focus shifts from thoughtful investigation and critique toward simply processing the volume, a shift detrimental to the whole research ecosystem. It's worth noting that even approaches like Hyperband automatic tuning for ANN models [Obtaining Irregular Learning Curves with Hyberband Tuned ANN model for Price Prediction] can generate multiple iterations of a single experiment, potentially contributing to the overall submission burden.

Ultimately, addressing this issue requires a cultural shift within the ML community. While imposing strict submission limits might be viewed as restrictive, a more nuanced approach—perhaps tiered limits based on reviewer experience or a system that rewards reviewers for providing detailed feedback—could be explored. The goal isn’t to stifle research but to ensure its quality and integrity. As AI continues to permeate every aspect of our lives, the need for rigorous, reliable research becomes ever more critical. The question moving forward isn’t whether we *can* continue to publish at the current rate, but whether we *should*, and how we can collectively prioritize quality over quantity in the pursuit of meaningful advancement.