Standard self-attention in transformers computes relevance between tokens by similarity, but it does not explicitly account for how much of a past state survives to the present. A new arXiv paper introduces Dynamical Kernel Attention (DKA), which instead derives attention from robust estimation of a dynamical process, as the title suggests: estimating the present from a record of the past.
According to the abstract, DKA is designed to address this gap by using a kernel-based approach that incorporates temporal persistence into the attention weights. The exact formulation is not fully visible in the excerpt, but the core idea is to replace pure similarity with a measure that reflects how past states propagate forward.
The significance is that this could make attention more sensitive to temporal dynamics in sequential data. However, the source is a single arXiv announcement with a truncated abstract, so no experimental results or comparisons are available in the provided text. Further details would require reading the full paper.