A new arXiv study tests a common guardrail in multi-agent reinforcement learning: permutation surrogates used to decide whether one agent's behavior influences another. In two games—a social dilemma and a coordination race—the authors construct pairs with zero directed influence by design and run 100 seeds. The standard procedure fails badly: if the non-stationary training transient is not excluded, it reports false positives in 100.00% and 99.95% of seeds, flagging agents that annealed exploration independently and never met as influencing each other.
Simply cutting out the transient is not enough. False-positive rates fall to 3.05% in the social dilemma but remain 11.8% in the coordination game. The authors' remedy changes the null model rather than the data: permuting the source within blocks of training time yields 5.25% and 5.50%, the only approach at or below the test's nominal size in both games. Block counts from 16 to 256 stay in the nominal region.
The paper also includes control experiments. A four-learner social dilemma defeats all three surrogates, and conditioning on identical pairs carries a cost of 15.20 points against 4.55 for the proposed test. Titrating injected links separates the block-permutation test from that conditioning (82.5% vs 58.0%), while an ablated test reports a positive whether or not a link exists, so its apparent sensitivity is uninformative. Code and data are released.