Large language models are increasingly used in computational social science to turn text into numeric scores for social and psychological constructs. But researchers rarely use the same rating scale, and it is unclear whether LLM-assigned scores stay comparable when the scale changes. A new arXiv preprint, "Change the Number, Change the Answer," directly tests this rating-scale sensitivity.
The paper's abstract frames the problem: LLM-based text scoring is becoming common, yet studies differ in how many points their rating scales have. The authors ask whether scores remain stable across these design choices. The title suggests their central finding—that altering the number format can alter the answer—though the full results are not detailed in the abstract.
Because this digest is based on a single preprint, there are no independent sources to compare or contrast. The main takeaway is that scale design is not a neutral choice in LLM scoring; researchers should treat scores as potentially scale-dependent.