Large language models are increasingly evaluated on expert-level mathematics and physics problems, but these benchmarks overwhelmingly focus on theoretical, symbolic reasoning. Real experimental data analysis presents a different kind of challenge, and that is precisely where the authors of a new arXiv preprint see a gap.
The paper introduces CME-SpectrumBench, a benchmark designed to test LLMs on condensed matter spectral data. By moving beyond abstract equations, it asks whether models can make sense of the kind of measured spectra that condensed matter physicists work with daily.
The abstract stops short of presenting results or model performance, so the benchmark's utility and difficulty remain open questions. Still, the proposal itself highlights an important shift: pushing LLM evaluation from clean theory toward the messy, concrete world of experimental data.