A new arXiv preprint, More Than Seeing: Vision as Scalable Anchors in Native Multimodal Pre-Training, investigates how visual signals affect the scaling behavior of models trained from scratch on both text and multimodal data. The authors frame vision not as a separate add-on but as part of a single model that learns language and visual inputs together.
The abstract argues that visual signals may serve as "scalable anchors" that support capabilities, but it does not yet present specific results or conclusions. Because the abstract is truncated, the paper's methods and findings are not available from this source.
The significance of the question is clear: most large models are trained primarily on text, with vision added later. This work asks whether starting with vision as a core part of pre-training changes how well the model scales. However, without the full paper, the answer remains open.