Vision-language models that accept multiple images often look competent in aggregate, but their accuracy hides where the pipeline actually breaks. A new arXiv preprint from Yiliu Xu, Xinjin Li, and Yu Ma isolates the failure point using a controlled schema-application task: models had to find two target characters in a 16-image sequence, identify their visual attributes, consult an item-specific visual codebook, and compose two ordered nonce code names.

To separate causes, the authors used a within-item 2-by-2 factorial oracle design. Oracle-A supplied a verified textual schema; Oracle-B supplied a verified ordered target-appearance representation. Both interventions kept the original images and final question. Across 72 new items and 1,188 model responses, Oracle-B improved strict exact accuracy by 54.6 percentage points (95% CI 46.3 to 63.0) and outperformed Oracle-A by 60.6 points. An auxiliary retrieval diagnostic reached only 49.1% ordered-appearance accuracy even after removing code-name generation, and providing both oracle sources yielded 100% accuracy across all models.

The results point to an instance-information bottleneck: the dominant recoverable deficit is upstream ordered instance recovery, not downstream schema use and composition. The authors' factorial oracle method offers a template for localizing failures in multi-image reasoning pipelines.