Conformal test martingales (CTMs) are sequential e-processes that test whether data are exchangeable. They work by feeding p-values derived from a conformity function A into a martingale; if exchangeability fails, the martingale is expected to grow. A new arXiv preprint (2610.23131) examines the limits of this approach from an information-geometric perspective.
The paper's central observation is that CTMs only see the data through A. Any distribution shift that preserves the push-forward measure P∘A^{-1} — that is, the distribution of conformity scores — leaves the p-value sequence unchanged. Such a shift is therefore indistinguishable from the original exchangeable process, no matter how large the change is in the raw data.
These blind spots are structural rather than implementation-specific. Because the test statistic is a function of A, information about the shift that is not encoded in the conformity score is inaccessible. The preprint frames this as an information-geometric limitation, suggesting that detecting such shifts requires either a different conformity function or a testing method that does not rely solely on score distributions.