Constrained Markov decision processes (CMDPs) are a common framework for policy optimization with multiple safety or performance constraints. A new arXiv paper introduces LOCU, or Lowdin-Orthogonalized Constraint Updates, to tackle a specific problem: when constraint gradients interact, the resulting update directions can become ill-conditioned, leading to numerically fragile steps.
The method applies Lowdin orthogonalization—a technique borrowed from quantum chemistry—to the constraint updates. This transformation is intended to decorrelate the gradients and produce more stable, well-conditioned update directions during optimization.
The abstract is brief and does not include experimental results or comparisons. It focuses on the problem formulation and the proposed algorithmic remedy, leaving open questions about empirical performance on standard CMDP benchmarks.