Deep reinforcement learning can produce brittle and unnatural gaits when legged robots are trained directly on joint torques, partly because inductive biases are hard to embed in such unconstrained learning. The authors propose a simple bias: latent actions extracted from a small dataset of expert demonstrations. This prior lets the policy leverage the structure of expert behavior while exploring more efficiently.
The latent action prior does not restrict the agent to the performance level of the demonstrations. According to the paper, the agent can still exceed those reward levels, and performance on transfer tasks improves significantly over baselines. This suggests the prior acts mainly as a good starting point rather than a ceiling.
Combining the action prior with style rewards for imitation yields the closest replication of the expert's behavior. The two mechanisms appear synergistic: the latent prior guides exploration, while style rewards refine the resulting gait pattern. The work, by Hausdörfer and colleagues, targets direct joint-level torque control and is scheduled for presentation at Humanoids 2026.