An arXiv paper, "Beyond Clean Accuracy: The Practical Challenges and Benefits of Pixel-Based Language Models," examines a class of models that read rendered text rather than relying on a fixed subword vocabulary. According to the abstract, this design offers a promising route for handling noisy and multilingual inputs, where tokenization often struggles.
However, the authors are careful to frame this as an open question. The abstract states that the practical value of pixel-based models depends on whether they can be reliably applied in real-world settings—suggesting that gains in robustness may come with trade-offs that are not yet fully understood.
Because the source is a single abstract, the article does not compare findings across studies. It simply reports the paper's stated motivation and caveat: pixel-based language models are an intriguing alternative, but their benefits are conditional on solving practical reliability issues.