Factorial survey experiments are a social science technique that presents participants with descriptions of socially complex situations, called vignettes, to elicit judgments such as social bias. The arXiv paper introduces this method as a tool for studying humans and large language models side by side, noting that it provides two major benefits—though the abstract truncates before enumerating them.
The paper's title poses a central question: when LLMs respond to such vignettes, do their judgments reveal genuine social bias, or are they shaped by social desirability—the tendency to give socially acceptable answers? This distinction matters for interpreting AI outputs as evidence of bias.
Because the abstract is incomplete, the specific benefits and any empirical findings are not available from the source. The work appears to be a methodological contribution, adapting a well-established human survey technique to evaluate AI systems in a more controlled and comparable way.