A new paper on arXiv, titled "Representations Have Trajectories," takes up a question that follows from a well-known observation: deep neural networks tend to learn simple, low-rank representations. The authors ask what this means for the way representations change over time during training. Their focus is on the temporal dimension of representation learning, which they describe in terms of trajectories.
The abstract is brief, but it signals a shift from studying static properties of learned representations to studying how those representations move and transform. By framing representations as trajectories, the paper suggests that the path a representation takes may be as important as its final state. This could have implications for understanding training dynamics, generalization, and the design of learning algorithms.
Because the source is a single arXiv abstract, the details of the methods and findings are not yet available. The paper's contribution, as stated, is to characterize these temporal dynamics in light of the low-rank simplicity bias, rather than to introduce a new training method or benchmark.