In high-stakes settings such as medicine and drug discovery, classification is often a means to an end: choosing the best action, not merely assigning a label. The authors argue that this requires proper class probabilities, but probabilities conditional on the raw features are computationally intractable in many important cases. Their alternative is to compute probabilities conditional on the trained classifier's output instead. This output-to-probability calibrator is cheap to build, needs to be computed only once, and can then be applied to all future outputs of the classifier.

In a simplified drug-discovery problem with a highly imbalanced dataset, the authors combined the calibrated probabilities with problem-dependent utilities through expected-utility maximization. They report that this always improved results across all tested utility sets, sometimes approaching the theoretical maximum. The one-time calibration also provides automatic uncertainty quantification for the calibrator itself, an expected-utility estimate for the augmented algorithm that can guide algorithm selection, and a generative mode that may help when the training dataset is biased.

Only one source was provided for this digest, so there are no conflicting findings to compare.