Question about NeurIPS discussion phase [D]

The recent Reddit post concerning a NeurIPS reviewer’s delayed score update highlights a persistent, and frankly frustrating, challenge in the peer review process: the disconnect between verbal acknowledgment of concerns and formal score adjustments. The author’s experience – a reviewer stating concerns were resolved, followed by a period of inaction and eventual score modification – isn’t unique. This situation echoes concerns raised in our previous piece, Conference Reviews: Asking Too Much?, where we discussed the burden placed on authors to extensively address reviewer comments, sometimes pushing the boundaries of the original submission. It also aligns with the broader issues of unresponsive reviewers, as detailed in No replies to rebuttals and comments even by AC, indicating a systemic problem with timely engagement and closure within the review cycle. The lack of consistent score updates following discussion phases undermines the integrity of the evaluation process and introduces unnecessary uncertainty for authors.

The core issue appears to be a lack of accountability or perhaps a misunderstanding of the reviewer’s role. While discussion phases are intended to clarify ambiguities and address concerns, they should ideally culminate in a revised assessment. The delay, as observed by the author, suggests a potential disconnect between the communicative aspect of the review and the formal scoring system. It’s possible that some reviewers view the discussion as a separate, informal process, not directly tied to their final score. This can be exacerbated by the sheer volume of papers reviewers handle, leading to rushed judgments or, in some cases, simply forgetting to update the score after a positive exchange. The author's eventual score adjustment, moving from a 2/4 to a more favorable rating, suggests the discussion *did* influence the reviewer's opinion, but the delayed action creates a sense of arbitrariness and unfairness. The fact that another reviewer remained completely silent further complicates the picture, demonstrating the inconsistency in reviewer engagement.

This isn't just about individual frustrations; it speaks to a broader need for structural improvements in conference review systems. Current systems often lack mechanisms to enforce timely score updates or to track reviewer responsiveness. While conferences are experimenting with different approaches, such as meta-reviews and more structured discussion protocols, a more robust system of accountability is needed. Perhaps a gentle reminder system, triggered after a certain period following a discussion phase, could encourage reviewers to finalize their scores. Furthermore, clarifying the expectations for reviewers regarding score updates during discussions would be beneficial. The ultimate goal should be to create a process that is both efficient and equitable, ensuring that authors receive timely and consistent feedback based on the entirety of the review process, not just fleeting conversations. The concern about reviewer behavior, as explored in NeurIPS 2026: Tips that might convince AC?, underscores the need for a more transparent and predictable system.

Looking ahead, it's crucial to monitor whether conferences adopt more proactive measures to address these issues. The increasing reliance on AI-assisted tools for manuscript screening and review management might offer opportunities to automate score update reminders and track reviewer responsiveness. However, it’s equally important to ensure that these technological solutions complement, rather than replace, the human element of peer review. The question remains: will conferences prioritize systemic changes to ensure a more accountable and efficient review process, or will these individual anecdotes continue to highlight a persistent flaw in the system?