Ari Herbet-Voss (Founder & CEO, RunSybil / Former OpenAI Researcher) delivers a deep dive into the realities of AI in offensive security, separating media hype from technical reality.
1. How LLMs Actually Work
Herbet-Voss clarifies that Large Language Models (LLMs) are not “reasoning engines.” They are essentially highly advanced “next-token predictors” (similar to a smartphone keyboard’s autocomplete) that use “few-shot learning” to mimic human reasoning based on patterns in their training data.
2. The Scaling Hypothesis
AI capabilities are governed by the scaling hypothesis: More Data + More Compute + More Parameters = Better Performance. Recently, AI models have shown “supralinear” (exponential) growth in reasoning capabilities. A model that is twice as large, trained on twice the data, can be four times as capable as its predecessor.
3. What is Getting Better with AI Scaling?
- Speed: The time it takes to go from discovering a vulnerability to creating an exploit has dropped drastically—from an average of 5 months to just 10 hours.
- Autonomy: Newer models require much less human “scaffolding” (guidance) to find vulnerabilities.
- Volume: AI can find a massive volume of “shallow” bugs incredibly quickly.
4. What is NOT Getting Better?
- Complex Bugs: The “capability floor” is not rising as fast as the ceiling. AI still heavily struggles to find deep, complex vulnerabilities that require holding state, temporal analysis, or concurrency (timing issues).
- Exploit Reliability: AI will generate thousands of potential vulnerabilities, but human experts are still required to filter, validate, and write the actual reliable exploits for them.
5. Implications for Security Teams
Herbet-Voss warns of the “million monkeys with typewriters” scenario. Because attackers only need an AI to get lucky once to breach a system, the sheer volume of AI-generated attacks puts defenders at a severe disadvantage. To survive, organizations must aggressively adopt AI to automate the patching of basic, shallow bugs so human defenders can focus on complex, state-level vulnerabilities.