Parameter-efficient continual learning aims to adapt a model to new tasks by updating only a small subset of adapter parameters, with the goal of limiting catastrophic forgetting. A common way to pick that subset is gradient attribution, which scores parameters by their importance to the current task. A new preprint on arXiv tests this approach within the B-matrix of LoRA adapters.
According to the abstract, the authors compared static gradient attribution with a random mask matched for density—that is, a randomly chosen subset of the same size. On loss-based forgetting, the random mask performed better than the static gradient attribution. The paper's title states the result directly: static gradient attribution underperforms a density-matched random mask.
Because this is a single preprint and the abstract is truncated, the underlying mechanism and experimental details are not fully available from the source. The finding, if it holds, would suggest that attribution-based selection is not always the best strategy for deciding which LoRA parameters to update in continual learning.