Self-supervised pretraining for handwritten text recognition (HTR) behaves differently from natural-image classification, according to an arXiv preprint. The paper reports that pixel reconstruction methods outperform contrastive methods for HTR, even though contrastive approaches are typically stronger in natural-image settings.
The authors argue that the difference follows from where the relevant information is located: handwritten text lives in the high-pixel-variance subspace. That is the subspace named in the paper's title, and the abstract presents it as the reason reconstruction-based pretraining wins for this task.
Only one source was available for this digest, so there is no independent result to compare. The abstract is truncated, meaning the full argument is not visible from the summary alone.