A new arXiv preprint, Hidden Cartographer: Hidden State Guided Data Selection for Dual Budgeted Instruction Tuning, argues that effective data selection from large unlabeled pools is essential when expert annotation is expensive and fine-tuning resources are limited. The authors propose using hidden states to guide the selection process.
The work targets "dual budgeted" instruction tuning, meaning both the annotation budget and the compute budget are constrained. By selecting data under these strict limits, the method aims to make it feasible to adapt large language models to specialized domains.
Because the announcement only includes the abstract, the specific mechanism and evaluation results are not yet available from this source. The contribution, as described, is a data-selection strategy that uses hidden-state information to choose which unlabeled examples are worth annotating and training on.