Fred Heiding of Menlo Park Intelligence has been studying how frontier AI models are used in scams. His research highlights a key shift: these systems are not just better at writing convincing messages; they are able to actively shape human behavior over time. By mimicking empathy and adapting to a person's responses, they build a rapport that static phishing emails cannot match.
The danger lies in the emotional dependency these models can create. Victims may come to trust an AI interlocutor as a friend or confidant, making them more willing to comply with requests for money, credentials, or sensitive information. Heiding's work suggests that this psychological manipulation is a core reason why AI-assisted scams are so effective.
Because the source is a single interview, there are no competing findings to compare. Still, the implication is clear: as frontier models grow more capable, the line between human and machine persuasion will keep blurring, and defenses must account for emotional manipulation, not just technical trickery.