AI coding assistants now write a large share of production code. But how secure is that code, and how consistent?
IOActive Senior Security Consultant Randy Flood answers that question at realigned.online, CanSecWest’s online AI security event for blue-team defenders.
Session: Security Roulette: How AI Code Generators Gamble with Your Applications
Date: Tuesday, November 10, 2026
Time: 10:30 a.m. PST (Track 2)
Format: Online
What the research found
Randy tested 27 AI code generators against 730 prompts written the way developers actually ask for code, with no mention of security. The results:
- The median configuration scored just 57.9% on security.
- Even the best performer, a security-tuned wrapper, still produced exploitable code in roughly 1 of every 8 cases.
- Authentication, cryptography, rate limiting, infrastructure-as-code, mobile, and CI/CD categories failed at rates between 70% and 99%.
- Running the same prompt through the same model produced a different security posture 30.4% of the time. Same model, same prompt: sometimes secure, sometimes exploitable.
Live demos
Every demo runs unmodified AI output through a working exploit, including:
- Cracking a hardcoded JWT secret in AI-generated auth code in under a second
- Exploiting AI-written XML parsers and SQL queries
- Draining a Solidity smart contract whose own comments flagged the reentrancy bug it shipped anyway
- Chaining two unrelated AI-generated files into a working token-theft attack
Why it matters
Even at the best configuration Randy tested, a 1,000-developer organization would still ship roughly 1,780 insecure AI suggestions per day. That’s more than any development team can review on its own. Randy closes with a practical defense framework built around automated scanning, expert security review of what automation misses, and red-teaming your own SAST pipeline.
All prompts, scores, and demo harnesses are publicly available in the AI Security Benchmark repository.
View the full schedule →
Read the session abstract →
Get tickets →
Need an independent assessment of the AI-generated code in your environment? Talk to IOActive’s AI/ML security team.