Summary
This report evaluates the security properties of code generated by large language models (LLMs) across various programming languages and known vulnerabilities. The primary objective is to quantify the security of AI-generated code without any security-specific guidance. The methodology involves a set of coding tasks designed to test four known vulnerabilities: SQL injection, cross-site scripting, log injection, and insecure cryptographic algorithms. The report details the performance of over 100 LLMs across four programming languages: Java, JavaScript, C#, and Python, using a total of 80 coding tasks. Results indicate that only 55% of the generated code is secure, revealing that a significant portion introduces known security flaws. The findings also suggest that security performance has not improved over time, and larger models do not consistently produce more secure code than smaller ones. The report outlines the implications of these findings for developers relying on AI-generated code.
