Large language models are increasingly used for code generation and analysis, but it remains unclear how deeply they understand program semantics. A new arXiv preprint investigates this by focusing on variable roles—recurring behavioral patterns such as accumulation, traversal, and keeping the best candidate seen so far. These roles are not syntactic features but reflect what a variable does in a program's logic.
The authors turn to Sajaniemi's taxonomy of variable roles, which was developed independently of machine learning research. By using this pre-existing framework, they avoid designing categories around what models already happen to capture. Their central question is which role distinctions can be recovered from the internal representations of language models trained on code.
Because the abstract does not report specific results, the paper's contribution at this stage is its framing: a principled, theory-driven probe of code understanding. The approach could help researchers identify where models fall short—for instance, confusing roles that are behaviorally distinct but syntactically similar—and guide efforts to build more semantically aware code models.