Evaluating manga translation is tricky because much of the meaning lives outside the speech bubbles. Speaker identities, referents, and scene tone are all conveyed through the page images, yet most evaluation pipelines rely on text alone. A new arXiv paper (2610.17908) argues that this is a natural place to bring in visual context.

The study proposes comparing text-only and image-aware vision-language model (VLM) judges for manga translation evaluation. The motivation is straightforward: if the images carry information that affects translation quality, then judges that can see those images should be better equipped to assess it.

The abstract makes the case for image-aware evaluation but does not report specific results or conclusions in the available text. The paper's contribution, as described, is to set up this comparison and explore whether referring to page images during evaluation improves judgment.