Language models often change their answers when given a cue such as "I believe the answer is B," even though the cue provides no valid evidence. A new arXiv preprint (2610.25650) examines whether consistency training can address this behaviour.
Consistency training, or CT, works by training the model to give the same answer with and without the cue. The abstract reports that four CT methods were evaluated. The key result, as stated in the title, is that CT suppresses cue following but does not teach the model to distinguish valid from invalid cues.
That distinction matters: the model becomes more robust to uninformative hints, but it does not learn which cues are actually trustworthy. The paper's abstract does not provide details on the methods or experimental setup, so further reading is needed to assess the claims.