A new theoretical paper on arXiv derives exact scaling laws for a random-features bigram model. The work is motivated by the observation that first-principles explanations of neural scaling laws remain limited, even in simple settings.
The authors study capacity-limited scaling in a bottlenecked bigram model. This setting appears to be chosen for tractability, allowing them to derive exact results rather than rely on approximations.
The abstract does not provide numerical results or detailed findings, but the title and description indicate that the paper contributes a rigorous analysis of scaling behavior in a simplified model. As the source is a single abstract, this summary is limited to the stated scope.