Referring expression comprehension (REC) usually asks a model to locate a single target described in a query. The authors of ReferFlow argue that complex queries often rely on intermediate visual references, where each step depends on prior grounding. Final-target evaluation, they say, leaves this process unexamined. To address this, they formulate trajectory-level REC, in which a model receives an ordered sequence of grounding steps built from one complex query and must localize the region associated with each step.
They introduce ReferFlow-Bench, which characterizes structured region transitions through four trajectory types: Refinement, Shift, Reset, and Mixture. Experiments with current models show that they struggle to ground intermediate references consistently across successive steps. The authors then propose ReferFlow, a model that separates interleaved text-latent trajectory generation from coordinate decoding. Stateful Region Latent Reasoning represents each target region as an object-indexed state of continuous latents, with explicit visual and spatial supervision to preserve region information for later grounding.
ReferFlow also uses Trajectory-Aware Parallel Decoding, which recovers all coordinate blocks in parallel through four geometry-constrained refinement rounds, with each block conditioned on its trajectory prefix. This reduces within-box sequential dependencies and removes cross-box autoregression. A Grounding-Aware GRPO objective promotes consistent grounding across steps and lets the model learn adaptive trajectory generation from complex queries. The authors report that ReferFlow achieves leading performance on general REC benchmarks and on ReferFlow-Bench.