Causal foundation models (CFMs) have shown competitive performance on standard benchmarks for estimating treatment effects at the individual level. A new preprint on arXiv, however, warns that this strength does not carry over when those unit-level estimates are combined for broader decisions.

The paper, titled "Unit-level aggregation breaks coverage in causal foundation models," argues that many real-world causal analyses rely on aggregated effects—such as average treatment effects across a population—rather than individual predictions. When CFM outputs are aggregated in this way, the resulting uncertainty intervals fail to achieve proper coverage, meaning they are systematically overconfident.

The authors do not dispute the unit-level accuracy of CFMs. Instead, they highlight a mismatch between the models' apparent precision and the reliability needed for policy-level conclusions. The finding suggests that practitioners should treat CFM uncertainty with caution when moving from individual estimates to aggregate decisions.

As a single preprint, this result has not yet been peer-reviewed, and the abstract provides limited detail on the experimental setup. Still, it points to a concrete limitation in how causal foundation models are evaluated and deployed. Further work will be needed to determine whether the coverage failure can be corrected or is inherent to the modeling approach.