ML conferences

Why paper limits still matter for fair AI research reviews

The push to keep papers short while demanding self-containment is creating a quiet crisis in ML conferences.

4 min readMachine Learning

The peer review process is supposed to be a quality filter, not a test of patience. Yet, as this author's experience shows, too many rejections hinge on a reviewer's comfort level with the material rather than the work's actual merit. The observation that theoretical papers are increasingly dismissed for being "difficult" or for not explaining terminology that is standard in the field is a sign of a deeper mismatch. We saw a similar dynamic play out in the recent NeurIPS cycle, where NeurIPS Acceptance Raises Questions About AI Review Justifications highlighted how score volatility and vague meta-reviews can feel arbitrary to authors. The problem isn't the reviewer's lack of knowledge, it's the lack of a shared agreement on what "self-contained" actually means when page limits are fixed.

The proposed rule, "Don't be a dick," is disarmingly simple, but it points to a structural fix that conferences have avoided. If the expectation is that a paper must be fully self-contained, then the only way to satisfy that is to spend precious pages on background material that a specialist would find redundant. That penalizes novel contributions in favor of pedagogical padding. The alternative, allowing reviewers to say, "I lack the prerequisite knowledge to assess this fairly, so I'll focus on the parts I can evaluate", would be more honest. It would also reduce the reviewer fatigue that comes from forcing someone to parse dense derivations they have no intention of using. The Securing a NeurIPS Ticket: Options for Authors from the Global South discussion shows that access and participation are already uneven; asking authors to anticipate every reviewer's baseline only deepens that inequality.

What stands out here is the shift in rejection reasons. A decade ago, the feedback was about related work or missing comparisons, things an author could address with effort. Now, the criticism is often about the failure to make the paper easier for a non-specialist to read. That's not a scientific objection; it's a preference for style over substance. The puzzlement is justified, especially since many ACs are professors who have likely written dense papers themselves. They know that real analysis is hard. They also know that no amount of rewriting makes it effortless. The NeurIPS Main Track: 7900 Submissions Accepted, 112 Oral Presentations numbers tell us the volume is only growing, which means reviewers are more pressed for time than ever. That makes the "review what you can" principle not just fair, but practical.

The takeaway for authors is not to write for the lowest common denominator, but to demand a review culture that meets them halfway. If you're a theoretical researcher, your job is to make your core idea accessible, not to re-derive linear algebra. If you're a reviewer, your job is to assess the contribution, not to penalize a paper for assuming you have a PhD. The next time you read a review that says "the math is hard," ask yourself: is it hard because it's wrong, or hard because it's not yet familiar? That distinction is the difference between a useful review and a lazy one. Watch for whether any major conference adopts language that explicitly allows reviewers to decline a paper on the basis of their own background, that would be the real signal of progress.

From Machine Learning

I've usually been commenting on threads on conference reviews. I'm now expressing my observations here.

To the best of my knowledge, paper lengths have been held constant at many conferences, and some conferences have "unlimited appendices" (e.g. NeurIPS / ICML / AAAI / ....) Historically, this was probably due to cost of printing for proceedings, but now, I suspect it's also to prevent reviewer fatigue.

Read the original at Machine Learning