AI companies sometimes alert law enforcement about user conversations they consider dangerous, even when no government agency has asked for the material. A new arXiv preprint examines these 'proactive referral' policies at major U.S. AI companies, asking when such voluntary disclosures are justified.
The central tension is that referral may expose people who only confided dangerous thoughts or feelings, not those who intended harm. The authors frame the issue through contextual integrity, a theory that asks whether information sharing respects the norms of the context in which it was originally provided.
Because the analysis draws on a single preprint, its findings are preliminary. But it highlights a growing policy question: as chatbots become confidants, companies need clear rules for when a private conversation becomes a police report.