An arXiv preprint introduces a method for fine-tuning language models with reinforcement learning by selectively updating attention heads. The authors propose ranking heads by gradient norm, then using that ranking to identify a small subset for sparse fine-tuning. The same ranking-and-partitioning framework is also applied to continual fine-tuning, where models must adapt to new tasks without forgetting old ones.

Because the abstract is truncated, details on the partitioning mechanism and experimental results are not available in the source. The approach appears aimed at reducing the computational cost of RL fine-tuning while preserving or improving performance.