A new paper on arXiv, titled From Observation to Intervention: Distilling Self-Generated In-Context Demonstrations via Executable Causal Intervention, addresses a key limitation in how large language models (LLMs) select their own few-shot examples. Self-generated in-context learning (ICL) allows LLMs to synthesize demonstrations on their own, and these often beat manually written prompts. However, the paper argues that current methods for choosing among these demonstrations are flawed.
The problem, according to the abstract, is that existing methods filter demonstrations by observation, assuming a rationale that may not capture true cause-and-effect. The authors propose shifting from observation to intervention: using executable causal interventions to distill which demonstrations actually drive the model's performance. This is a conceptual move from correlational selection to causal selection.
The abstract is truncated in the source, so specific experimental results and implementation details are not available in the snippet. Still, the core contribution is clear: a new framework for evaluating self-generated demonstrations based on causal intervention rather than passive observation. The paper is available on arXiv under identifier 2610.21149.