The authors propose DAGFormer, a Transformer language model whose cross-layer connections are planned by a jointly trained topology predictor. In a standard Transformer, information flows through residual connections. DAGFormer adds direct connections between layers, and the predictor decides where these connections should go.

The abstract does not report experimental results, so it is unclear how DAGFormer compares to standard Transformers on downstream tasks. The work is described as a way to plan cross-layer connections globally, rather than relying on fixed residual paths.

The paper is available on arXiv as a preprint.