A new systematic literature review from arXiv examines the security implications of using large language models in software development. The review finds that while LLMs significantly accelerate coding tasks, they also introduce severe security vulnerabilities that developers must contend with.

The study also investigates the limitations of current security benchmarks used to evaluate these risks, suggesting that existing tests may not fully capture the range of potential weaknesses. In addition, the review considers the role of human developers, indicating that human oversight remains a crucial factor in identifying and addressing security issues that automated tools miss.

Taken together, the findings point to a need for more robust evaluation methods and for keeping humans in the loop when AI-assisted coding is deployed in real-world projects.