Robot foundation models may one day be steered at a finer grain than whole behaviors. A new arXiv preprint asks whether better feature discovery can improve control through internal interventions and through the training data itself.
The motivation comes from recent work in vision and language models, which has revealed that learned features are multidimensional rather than single directions in activation space. That observation leads the authors to consider block-sparse featurizers, a method aimed at capturing such structure.
The abstract provided ends before describing outcomes, so no experimental results or comparisons are available from this source. The contribution, as stated, is the question and the proposed featurizer direction.