Layer normalization is usually studied as a training stabilizer, but this paper treats it as a multivariate nonlinearity between affine layers. The authors ask how the arrangement of normalization units affects what a network can represent as width grows.

They prove that a shallow network with a single LN unit is not universal, no matter how wide. In contrast, if normalization is applied independently to parallel groups—a "PLN-Net"—the network becomes universal as the number of groups increases. The proof extends to deep networks, RMSNorm, position-wise feed-forward blocks, and convolutional networks under suitable grouping.

Experiments on MLPs and physics-informed neural networks support the theory. CNN and ViT tests show that group count influences the finite-sample fitting envelope, but other factors—which axes are grouped, where normalization is inserted, and the label structure—also shape fitting and generalization.