A new arXiv preprint investigates how large language models represent code. The authors note that a given computation can be written in many programming languages and implemented through different strategies. They ask whether LLMs capture the underlying commonality.

The abstract reports that LLM activations do capture this shared structure. The representations are described as reusable and, based on the paper's title, composable. This suggests the models may build abstract, language-agnostic notions of computation rather than merely memorizing syntax.

The full details are not available in the abstract, which is truncated. Still, the finding points toward a deeper understanding of how LLMs generalize across code, with potential implications for program synthesis and translation.