Contrastive reinforcement learning (CRL) learns goal-conditioned values by associating state-action pairs with the states subsequently reached. This relies on the assumption that no unobserved variable jointly affects the recorded action and its outcome. A new paper on arXiv proposes counterfactual rollouts for offline contrastive goal-conditioned RL, directly addressing this assumption.
The method is designed for offline settings, where data is fixed and the standard assumption may be violated. The paper's title and abstract indicate that counterfactual rollouts are introduced to handle this scenario, though the full details are not provided in the abstract.