The paper introduces a family of Rényi-based concentration metrics, called α-RC, for analyzing the internal geometry of multilayer perceptrons. The authors work within the Kolmogorov-Arnold Geometry framework, which studies how a hidden layer responds locally to input changes by examining the minors of its Jacobian. The key question is whether a small number of minors dominate the rest, a property known as concentration.
The existing Participation Ratio metric has a major limitation: its maximum value depends on the network's size, so raw values cannot be compared across architectures. The α-RC metrics address this by measuring the Rényi divergence of order α between the normalized minor magnitudes and the uniform distribution. All metrics in the family are bounded between 0 and 1, where 0 means the magnitude is spread evenly and 1 means a single minor carries all the weight.
The metrics are non-decreasing in α, and at α=2 they reduce to an explicit transformation of the Participation Ratio, connecting the new approach to the old one. The authors validate the proposal with computational experiments on single-hidden-layer MLPs trained on a synthetic dataset, reporting that the new metrics offer clear benefits for interpretability analysis.