Struggling to reproduce paper results before improving them — stuck below reported accuracy [R]

The reproducibility crisis in AI research is more than an academic inconvenience; it is a structural challenge that undermines the very foundation of scientific progress. When a PhD student cannot match a published baseline after exhaustive effort, the incident reveals a systemic gap between how research is presented and how it can actually be verified. This is not an isolated experience but a widespread phenomenon that affects researchers at every level, from graduate students to seasoned practitioners. The frustration expressed by the original poster reflects a reality many in the field have encountered: the gap between reported results and achievable results can be substantial, and the reasons for that gap are often opaque.

The core issue stems from what might be called documentation asymmetry. Researchers who publish papers operate under pressure to present novel contributions, but the granular details required for exact reproduction are frequently treated as supplementary rather than essential. Hyperparameters, random seeds, data preprocessing pipelines, and evaluation protocols are often described in broad strokes, leaving reproducibility to chance or extensive reverse engineering. When authors are unresponsive to clarification requests, as in this case, the researcher is left to fill in gaps through inference and experimentation, a process that is both time-consuming and intellectually taxing. The community benefits when authors treat reproducibility as a core component of their contribution, not an afterthought.

For researchers facing this challenge, there are practical paths forward that do not require abandoning the original goal. First, treating the 73% result as a legitimate baseline for experimentation is a valid approach; the improvement pipeline can still be evaluated relative to that starting point, and the resulting delta can be reported transparently. Second, exploring alternative implementations, such as open-source repositories that claim to reproduce the paper, can provide additional reference points and may surface details omitted from the original text. Third, engaging with the broader community through forums and discussions can yield insights from others who have attempted similar reproduction efforts. The key is to document every decision and deviation meticulously, so that future researchers building on this work have a clearer roadmap.

Looking ahead, the AI research community is gradually recognizing that reproducibility must be engineered into the research process, not left to chance. Initiatives around standardized benchmarks, code release requirements, and detailed reporting checklists represent meaningful steps toward a more transparent ecosystem. However, cultural change takes time, and individual researchers must navigate the current landscape while advocating for better practices. The question worth watching is whether emerging tools and platforms can automate aspects of reproducibility reporting, making it easier for authors to share the complete context of their work. Until then, researchers like the original poster will continue to face these challenges, and their willingness to ask hard questions about reproducibility is itself a contribution to the field.