A new arXiv paper, ControlJEPA, takes aim at a blind spot in next-token prediction models. According to the abstract, these models leave hidden-state trajectories largely unconstrained, and existing geometric regularizers primarily control the direction of trajectory deviations rather than their magnitude. That gap can allow hidden states to drift in ways that are not captured by standard training objectives.
The paper introduces ControlJEPA, a method that applies a Lyapunov tube loss to regularize trajectories in a principled manner. The Lyapunov-based approach is designed to keep deviations within a bounded tube, addressing both direction and magnitude. This is a departure from earlier geometric regularizers, which focus on angular alignment.
The abstract is brief, so the full details of the loss formulation and experimental results are not available from the source alone. The paper is posted on arXiv under the identifier 2610.24951, and the abstract suggests the work is aimed at making hidden-state dynamics more stable and predictable during training.