The quiet danger in agent memory isn't that it fails, but that it succeeds too faithfully. This small case study on procedural-memory poisoning makes that plain: replaying recorded edits reintroduced a planted bad line into 30 out of 30 repositories, with every test still passing. A hidden verifier with 187 behavioral checks caught nothing because the checks weren't looking for the poison. The tests passed, the edits matched, and the agent happily repeated a mistake it never knew it was making.
That result should reframe how we think about skill retention in coding agents. We often treat memory as a neutral archive, a simple recording of what worked. But memory is also a recommender system, and naive retrieval will surface whatever looks most relevant, including the one session that slipped a bad line into an otherwise good fix. In this study, naive memory exposed the poison in 3 of 30 sessions. Not catastrophic, but not clean. The more interesting arm, proof before reuse, served a skill only after two later sessions reproduced the same change and passed their tests, and showed the agent only the lines those sessions also wrote. It exposed the poison in 0 of 30 sessions, including the fused case where a step-level check passed and only line-level filtering removed the bad edit.
We should be honest about what this does and doesn't prove. The sample is small, one task, five sessions per cell, and the p-value sits around 0.24, which means the difference between arms isn't statistically significant. No model actually applied a poison it saw, so this measures exposure, not harm. And a poison placed directly in the repository defeats the method entirely, a failure the authors openly document. This is a case study, not a verdict. But it is a useful one, especially alongside the broader problem of evaluating agents without ground truth, a challenge our own coverage has explored in Assessing Coding Agents Without Ground Truth via Consistency Mapping. The verifier here is strong, 187 behavioral checks, yet it still couldn't flag a poisoned line that preserved test outcomes. That's a reminder that verification and memory are separate failure surfaces.
The open question the authors pose is the one worth sitting with: does line-level reproduction actually signal trust, or does it just reward whatever clean sessions happen to repeat? If three clean sessions all make the same incidental edit, that edit becomes a skill, regardless of whether it matters. The filter removes poison that appears once, but it also canonizes habits that appear often. That's not a flaw in the method so much as a definition of trust, and it deserves scrutiny. For practitioners, the practical takeaway is direct: if you're building agents that learn from past sessions, don't assume passing tests are enough to validate a learned skill, and don't assume replaying recorded edits is safe. The replay arm here put the poison back in every single repository. That's not an edge case; it's the default behavior of a system that treats memory as a tape recorder. The proof-before-reuse approach is more conservative, and conservatism is exactly what you want when the cost of a bad habit is silent repetition. This also echoes the frustration of rigorous work getting dismissed too quickly, a theme we raised in Desk Rejection in Two Days: When Rigorous Work Meets Silence. Small, transparent studies like this one deserve attention not because they settle the question, but because they give the rest of us something concrete to build on. Watch the line-level filter. It's the most fragile and the most revealing part of the design, and it will likely determine whether this approach scales beyond a single task.