Recurrent-depth language models improve reasoning by reusing a shared network to refine internal representations, but this extra recurrence makes inference more expensive. A new arXiv paper proposes RAMP (Recurrent Adaptive Mixed Precision) to reduce that cost during the prefill phase.
The method applies adaptive mixed precision, adjusting numerical precision to cut computation without necessarily sacrificing quality. The abstract notes the overhead problem but is cut off before presenting experimental results, so the write-up is limited to the motivation and the proposed approach.
RAMP is aimed specifically at prefill, the initial processing of input tokens, which is a distinct stage from the later generation loop. Further details on implementation and measured outcomes are not available from the source.