When Your Reproduction Doesn't Match the Published Results

Reproducibility in machine learning is often treated as a procedural formality, but the experience of this researcher reveals it as a genuine test of scientific integrity. When you run an author's released code with their exact hyperparameters across multiple seeds and months, and a gap persists on two benchmarks, that is not a mistake, it is a signal. The researcher has done everything right: kept the code unchanged, verified other benchmarks match, and repeated the experiment four times. The burden now falls on the review process, which too often defaults to trusting the published number over a careful reproduction. This is a moment where the community must decide whether it values transparency or convenience. For readers managing their own data workflows, this tension is familiar, whether you are reconciling Discover smarter workflows from the week's top Excel community insights or debugging a pivot table that refuses to cooperate, the principle is the same: your observed results deserve honest documentation, not forced alignment with someone else's published claim. The researcher's plan to report both numbers, clearly labeled, is the correct path. Reviewers should accept that a published baseline is not a sacred number, it is a claim that can be tested. The researcher should contact the authors first, not to accuse, but to share findings and ask if they have additional context. Mentioning that contact in the paper signals good faith and thoroughness. The wording matters: frame it as "our reproduction using the authors' released code and specified hyperparameters" rather than "we found a discrepancy." This is not an attack; it is science working as intended. The real question is whether reviewers will accept a "published vs. reproduced" row in the main table. They should. If a baseline's published numbers cannot be consistently reproduced, that is relevant context for every comparison in the paper. Pushing it to an appendix buries the finding. The researcher's results do not look weak next to the published numbers, they look honest. That is a stronger position than inflated baselines. One concrete takeaway: if you are in this position, include both numbers in the main table, label them clearly, and cite the GitHub issue or author correspondence in the paper. Reviewers who reject that are rejecting reproducibility itself. For those who Stop letting data problems masquerade as pivot table problems, the lesson is the same, the method matters more than the number.