Most Transformer sequence generators produce tokens from left to right. A new arXiv preprint argues that this assumption may be a poor fit for sequence labelling, where dependencies between tags can run in other directions. The proposed model, TagInsert, instead learns a dynamic ordering for generating labels.

TagInsert is designed for low-resource tagging scenarios. The authors say that in such settings, dependencies may not naturally align with a left-to-right order, so allowing the model to choose its own generation order could be beneficial. The abstract does not include experimental results or comparisons, so the claims are currently presented at the proposal level.

Because the article is based on a single source, there are no independent results to corroborate the approach. The preprint's significance lies mainly in questioning a default assumption in Transformer-based sequence generation and offering an alternative for tasks where ordering is not fixed.