Google Research has published a blog post describing a method to accelerate Gemini Nano models on Pixel devices. The technique, called frozen multi-token prediction (MTP), addresses the usual bottleneck in autoregressive generation, where models emit one token at a time. By predicting multiple future tokens in a single forward pass, the model can produce longer sequences with fewer steps, directly lowering latency.
The "frozen" part refers to the training scheme: the base Gemini Nano model's weights are not updated. Instead, a small auxiliary prediction head is trained to forecast several tokens ahead. This keeps the original model intact while adding a lightweight component that can be used during inference. The blog reports that this approach yields meaningful speedups on Pixel hardware, with only a small trade-off in output quality.
Because the base model remains frozen, the technique is compatible with existing deployments and does not require expensive full retraining. The research is presented as a practical optimization for on-device AI, where responsiveness is critical. While the post focuses on Pixel, the underlying idea could apply to other edge devices running similar models.