Regularization-based continual learning tries to avoid catastrophic forgetting by protecting each network parameter in proportion to an estimate of how important it is. A new arXiv paper (2610.14258) points out a mismatch: those estimates come from finite samples and heuristics, yet the algorithms use them as if they were exact.

The paper reports four studies on this problem. Its title identifies two reasons why importance estimates shrink over the course of learning, though the abstract does not spell out what those reasons are or what the four studies found. Because protection strength is proportional to the estimate, shrinking estimates would mean progressively weaker protection for parameters the model may still need.

The authors frame this as a caution about relying on importance estimates in continual learning. Without the full text, the specific mechanisms remain unclear from the abstract alone.