BehaviorFormer, described in a new arXiv preprint, tackles the problem of discovering behavioral patterns in open-ended games. The authors note that these environments are challenging because they lack clear objectives or ground-truth annotations, making it hard to define what a meaningful behavior is.
The proposed model extends a next-token prediction Transformer with a deep clustering algorithm. This combination is intended to let the model learn representations that are both predictive and naturally grouped into distinct behavioral patterns, without requiring manual labels.
The abstract does not include experimental results or comparisons, so the effectiveness of the approach is not yet detailed. The work is presented as a method for unsupervised discovery of structure in complex game-play data.