A new arXiv preprint, "Social Priors: Descriptions of AI Users Shape How Language Models Trust and Cooperate," investigates a feedback loop in AI development. The authors note that descriptions of how people use AI are increasingly folded into the training data for the next generation of models. Their central question is whether what a model reads about its users changes how it treats those users in practice.
To explore this, the researchers employ contrastive synthetic-document finetuning. This approach likely involves training models on deliberately contrasting descriptions of users, then measuring differences in downstream behavior such as trust and cooperation. The abstract frames this as a way to test whether social priors—preconceptions formed from text—can be induced in language models.
Because the abstract is truncated, the specific results and conclusions are not available in the source. The significance lies in the premise: if model behavior shifts based on user descriptions in training data, then the way people are written about online could indirectly shape how AI systems treat them. The preprint is a single source, so no independent comparisons are possible here.