Reinforcement learning (RL) is a common way to fine-tune vision-language models (VLMs) for agentic tasks, but the reasons for its success have been poorly understood. A new preprint investigates this question across four different agent environments, asking whether RL improves the model's visual perception or simply changes where it looks.
The authors find that RL primarily teaches the model where to attend, not how to perceive. In other words, the model learns to focus on the right parts of an image, but its underlying visual representation remains largely unchanged. This distinction is significant because it suggests that current RL pipelines may be leaving perceptual capabilities on the table.
If RL only adjusts attention, then improving a VLM's perception may require different training signals or architectural changes. The study highlights a potential limit of RL-based fine-tuning and points toward a more nuanced view of what these training methods actually contribute.