Arabic calligraphy is a high-stakes cultural visual domain for Muslim communities, yet vision-language model (VLM) evaluations rarely test its fine-grained script traditions. A new preprint on arXiv audits 12 VLMs to address this gap, using a cleaned benchmark of 1,195 images spanning six calligraphy styles.
The abstract describes the benchmark as a six-way classification task with numeric evaluation, but the text is truncated before results are reported. The authors' motivation is clear: standard VLM benchmarks tend to ignore specialized visual domains that carry deep cultural significance.
By introducing this benchmark, the study argues that fine-grained Arabic calligraphy recognition deserves more attention from the VLM research community. The work also serves as a reminder that evaluation sets should reflect the diverse, real-world visual contexts where these models are expected to perform.