Reinforcement learning with verifiable rewards has proven effective for mathematical reasoning when a final answer can be checked automatically. But a correct answer does not certify the derivation behind it, and proof problems offer no answer to check at all. That leaves a significant gap in training models to reason soundly.

The new work addresses this by splicing hints into reasoning traces, drawing on demonstrator failures. Instead of relying solely on outcome verification, the method uses information about where a demonstrator's reasoning went astray to guide learning. These hints are inserted into the trace, giving the model corrective signals even when no final answer exists.

This approach is notable because it targets the mismatch between answer correctness and derivation quality. By learning from failures rather than only successes, it offers a path toward more robust reasoning in settings where verification is partial or absent. The paper is available on arXiv under identifier 2610.13996.