Medical image segmentation models are increasingly paired with uncertainty estimates, but how those estimates are judged may miss the point. According to a new preprint on arXiv, the field usually relies on calibration error or pixel-level error-detection scores. The authors argue that these metrics do not show whether uncertainty actually improves downstream review decisions.

The paper proposes a decision-aware evaluation protocol, though the abstract released so far gives only the outline. The core idea is to evaluate uncertainty by its effect on the decisions a clinician or reviewer makes after seeing the segmentation, rather than by statistical properties of the uncertainty values alone.

Because the abstract is truncated, the specific design of the protocol is not yet available from this source. The preprint is arXiv:2610.21616v1.