Let's be direct: chasing perfection in stenosis detection is the wrong bet. The goal should be recall, even if it means inviting a few false positives. That's not lowering standards, it's aligning your metric with the reality of clinical workflows, where missing a narrowing carries far more risk than flagging one that turns out to be nothing.
The engineer behind this project has already hit a wall that many of us know well: a model trained on one annotation scheme fails on another. Same disease, two different tasks because one dataset used fixed-size boxes around a point while another fitted boxes to the lesion. That's not a model problem; it's a data problem. It echoes the same kind of fragmentation we've seen in other fields, like when Harbor Brings Isolated MicroVM Sandboxes to Agent Evaluations showed that evaluation consistency matters as much as the model itself. Here, the inconsistency is baked into the ground truth, and that's where the real work begins.
The proposed approach, using vessel segmentation and centerlines to build a radius profile, then filtering candidates with a network, is sensible. It leans on geometry before complexity, which is exactly what you want when your training signal is noisy. But the evaluation question is harder, and more honest. Doctors don't mark every narrowing. So any recall computed against those labels is optimistic, maybe dangerously so. This is exactly the kind of challenge that Speed up your 500-image annotation without sacrificing model accuracy tackles head-on: annotation speed and quality are trade-offs you have to manage deliberately, not ignore.
So how do you get an honest number? You don't. Not by relying on incomplete labels. Instead, build a pipeline that treats the radiologist's mark as a floor, not the truth. Use the radius profile to generate candidates, have a second expert review only the false positives, and call that your upper bound. The gap between the two numbers is the uncertainty you live with. That's not a failure, it's a design constraint. And designing around constraints is what separates a useful tool from a research artifact.
The open question worth watching is whether the vessel profile plus network pipeline can keep false positives low enough, not zero, but below the threshold of annoyance, while maintaining the recall that actually matters. That threshold is where trust gets built, and no public dataset will tell you where it is for your own clinical partners.