Fine-grained visual classification (FGVC) asks models to tell apart categories that look nearly identical, such as different bird species or plant varieties, often with only a handful of labeled examples. Multimodal large language models (MLLMs) have strong fine-grained perception and broad open-world knowledge, but fine-tuning them on sparse samples tends to overfit and erode that knowledge. The authors of UniFGVC argue that parameter fine-tuning is the wrong paradigm for this setting.
Instead, UniFGVC reframes few-shot FGVC as a retrieval problem. Each category is represented by a predefined template, and an image is classified by computing similarity between its features and all templates. The central component is the Category-Discriminative Visual Captioner (CDV-Captioner), which uses an MLLM to produce structured textual descriptions that highlight the discriminative attributes separating similar classes. Chain-of-thought reasoning and visual references are incorporated to reduce hallucination and strengthen the captions' discriminative power.
The image and its generated caption are then encoded with arbitrary pre-trained visual and textual encoders in a plug-and-play manner, so the framework does not require task-specific training. Across 12 FGVC datasets, UniFGVC consistently outperformed existing few-shot CLIP-based methods and also surpassed several fully supervised MLLM-based approaches, supporting the claim that retrieval-based inference is a general and effective alternative to fine-tuning in the low-data regime.