Researchers and practitioners are turning to large language models (LLMs) to automate the detection of vulnerabilities in code. A new preprint on arXiv examines whether these models are influenced by the same cognitive shortcuts that affect human reviewers. The abstract notes that recent work has already shown LLMs can be susceptible to such heuristics, and this paper investigates that phenomenon in the context of code vulnerability detection.
The study's framing suggests that LLM-based detection may not be purely rational or rule-driven. Instead, the models might rely on patterns that mirror human cognitive biases, potentially leading to systematic errors. The authors appear to be probing how these heuristics manifest and what they mean for the trustworthiness of automated security tools.
Because this is a single preprint with a limited abstract, the specific heuristics studied and the experimental results are not detailed here. Still, the core claim is clear: LLMs are not neutral analyzers, and their vulnerability detection performance may be shaped by the same mental shortcuts that influence human judgment. This has practical implications for security workflows that increasingly depend on LLM assistance.