Machine learning models increasingly handle structured data of varying sizes, such as sets and graphs. A key challenge is size generalization: a model trained on inputs of one size must perform well on inputs of another size. This paper from arXiv introduces a theoretical framework for understanding this problem.
The authors derive scaling laws that are dimension-free, meaning they do not depend on the input dimension. They also demonstrate an exponential separation between different model architectures, suggesting that some approaches generalize far more effectively than others when input size changes. The findings provide a foundation for designing models that can robustly handle variable-size data.