Most neural networks are built with a fixed size chosen before training, which separates architecture selection from weight learning. A new preprint introduces the Neurogenesis Network (NGN), a differentiable parameterization that treats network size itself as a learnable quantity. Rather than choosing how many ordered components—likely parameters or layers—in advance, NGN lets the network adjust its own size during optimization.
The core idea is to represent count as a continuous, differentiable variable, allowing gradients to flow into decisions about network capacity. This unifies the two traditionally disjoint stages of model design: deciding how large the network should be and then fitting its weights. The abstract does not yet disclose experimental results, so practical benefits remain to be demonstrated.
As a single-source digest, we note that the claim of joint optimization is stated directly in the abstract, but no benchmarks or comparisons to existing methods are provided. The contribution is primarily conceptual: a mechanism for learning the number of ordered elements in a network without discrete search.