Large language models are increasingly used as synthetic survey respondents, offering a cheap way to generate human-like answers at scale. But most existing evaluations, the authors of this paper argue, only check whether individual answers look plausible — whether they read like something a person might say. That, they contend, is a weak test.

The paper instead asks a stricter, psychometric question: do LLMs preserve the properties that make survey responses valid for measurement and inference? The title gives away the answer: plausible but not valid. The authors argue that surface-level believability does not guarantee that the responses maintain the measurement qualities required for valid research.

The abstract is truncated in the source, so the specific tests and datasets are not described here. The core claim, however, is clear: the field should move beyond plausibility checks and hold synthetic respondents to the same psychometric standards applied to human survey data.