Vision Transformers (ViTs) rely on positional embeddings to encode where each image patch sits, which lets the model tell apart different spatial layouts of visual content. A new preprint on arXiv takes a closer look at how this works internally, comparing two widely used encoding schemes: absolute and rotary positional encodings.

The authors note that while these embeddings are known to be essential, the field still understands little about how the choice of encoding influences the model's internal representation of position. The paper offers a mechanistic comparison of the two approaches, probing how each one shapes the way ViTs process spatial information.

As a preprint, the study has not yet been peer-reviewed. The abstract does not include specific results, but the work points to a more detailed understanding of a core component in vision models.