Over-parameterized linear regression has been a major focus in machine learning theory, but typical analyses rely on simplifying assumptions. According to a new arXiv paper, most existing work assumes covariates are independent and covariance matrices are non-degenerate. The authors argue that this leaves an important gap.
The paper, titled "The Zero Pattern of a Design Matrix Drives Multiple Descent in Over-parameterized Regression," instead examines how the structure of the design matrix—specifically its zero pattern—affects the phenomenon of multiple descent. The abstract suggests that this zero pattern is a primary factor, rather than the covariance properties that have dominated prior analyses.
Because the abstract is brief, the full details of the theoretical results are not yet available from the summary alone. But the claim is clear: the specific arrangement of zeros in the design matrix can drive the multiple-descent behavior observed in over-parameterized models.