Conformal prediction provides prediction sets with finite-sample coverage guarantees, and Mondrian calibration extends these guarantees to predefined groups. But the authors of a new arXiv preprint argue that such guarantees are coarse: even when each group is covered at the promised rate, finer strata inside a group can still experience large disparities.

The paper introduces propensity standardization to assess how the distribution of groups across strata affects aggregate comparisons, showing that group composition can drive disagreement between aggregate and finer-stratum results. To mitigate the problem, they propose Tilted Mondrian, a calibration adjustment that improves fine-grained fairness without additional model training.

Tilted Mondrian retains the original groupwise coverage guarantees while reducing disparities in both coverage and prediction set size within finer strata. The authors also derive a method-agnostic bound that quantifies how much reduction in fine-grained coverage disparity is achievable.