A new paper on arXiv investigates whether large language models' memorization of training data is context-sensitive. The authors focus on prefix-based extraction, where a model is given a prefix from a training example and then assigns high probability to the original continuation. This is a common way to probe for memorized sequences.
The paper argues that standard evaluations often use isolated prefixes, but deployed systems rarely present such clean inputs. Instead, real-world queries are embedded in surrounding context, which may change whether memorized content is elicited. The abstract suggests the work moves beyond isolated prefixes to examine how context affects extraction.
Because the source is a single preprint abstract, the full findings are not yet available. Still, the research points to a practical concern: if memorization is context-sensitive, current privacy assessments may underestimate or mischaracterize how LLMs leak training data in actual use.