World models that predict future visual states typically need action labels, but those labels are scarce and differ across robots, games, and other environments. OneLA instead learns continuous latent actions directly from video, using a shared orthogonal dictionary and a single predictive objective. The design avoids task-specific components and cross-domain alignment losses during pretraining, so the same latent space can serve multiple domains.

Once pretrained, the latent actions act as a common conditioning interface. Lightweight adapters translate native control commands into this space, and the world model predicts context-dependent transitions conditioned on the current visual state. The authors report that OneLA improves action readout across domains and reduces action cycle-transfer rotation error by 24.8% compared with the strongest evaluated baseline. In a joint-training comparison, it also achieves a 20.5% higher Human Preference Score v3 for real-scene generation.

These results point toward general-purpose world models that can learn from diverse video and be controlled flexibly through a latent action space. The paper is available on arXiv.