Datamodels are tools that predict how a model's behavior changes when parts of its training data are included or removed. They are useful for understanding which training examples influence specific predictions. However, existing linear datamodels assign a fixed contribution to each training unit, which simplifies the attribution problem but misses important interactions.
The new paper argues that this linear approach cannot represent non-additive joint effects—situations where the influence of a group of training examples is not simply the sum of their individual influences. To capture such interactions, the authors propose tensor network datamodels, a more expressive framework that can model higher-order relationships in the training data.
By moving beyond fixed per-example contributions, tensor network datamodels aim to provide a richer and more accurate picture of how training data shapes model behavior. The abstract highlights this as a key motivation, though the full details of the method and its empirical results are not included in the available snippet.