A preprint on arXiv examines whether automated discovery systems can trust the explanations they generate. These systems propose scientific explanations and run tests to decide which to accept. The paper asks a pointed question: does passing a test actually establish the intended claim, or merely rule out some alternatives?
The authors introduce four executable operations for changing physical descriptions, apparently as null controls. These operations are meant to test whether a proposed explanation is genuinely supported, rather than just consistent with the data. The title summarizes the central finding: null controls are necessary but not sufficient for validating scientific discovery and interpretability.
The work highlights a gap in current evaluation practices. Even when an automated system's output passes a guard, that success may not be enough to confirm the explanation's validity. The paper suggests that stronger, more nuanced checks are needed before accepting machine-generated science.