My Personal Garden

Ari Herbet-Voss (Founder & CEO, RunSybil / Former OpenAI Researcher) delivers a deep dive into the realities of AI in offensive security, separating media hype from technical reality. 1. How LLMs Actually WorkHerbet-Voss clarifies that Large Language Models (LLMs) are not "reasoning engines." They are essentially highly advanced "next-token predictors" (similar to a smartphone keyboard's autocomplete) that use "few-shot learning" to mimic human reasoning based on patterns in their training data. 2. The Scaling HypothesisAI capabilities are governed by the scaling hypothesis: More Data + More Compute + More Parameters…

The "Agentic Age" of Vulnerability Research (Yan Shoshitaishvili)Professor Yan Shoshitaishvili argues that cybersecurity has moved beyond traditional methods and simple Large Language Models (LLMs) into the "Agentic Age," where autonomous AI workflows are fundamentally changing how software bugs are found. The Problem with the Old Way: Traditionally, finding vulnerabilities was a slow, human-intensive academic process. Researchers would develop new static or dynamic analysis tools (like fuzzers), which required massive manual effort and struggled to scale. The Problem with Raw AI: Simply pasting code into an LLM (like ChatGPT) and asking…

Security researchers from Tencent Zhuque Lab present their findings on vulnerabilities within the Model Context Protocol (MCP), a "universal connector" designed to link AI agents with external tools and real-world data. The core of their research highlights a paradigm "threat shift": attackers are moving away from exploiting traditional software code vulnerabilities to exploiting the context window of Large Language Models (LLMs). This is primarily achieved through Indirect Prompt Injection, taking advantage of the fact that AI agents struggle to distinguish between "content to be read" and "instructions to…

"Make Agent Defeat Agent: Automatic Detection of Taint-Style Vulnerabilities in LLM-based Agents" by Ke Li and Fengyu Liu at Black Hat Europe 2025: The Core ProblemLarge Language Model (LLM) agents are increasingly popular, but they introduce new security risks. Agents act on natural language prompts to perform complex tasks by invoking external tools, running code, or accessing databases. The speakers highlight that developers often blindly trust the structured output of LLMs without proper validation or sanitization. This leads to "taint-style vulnerabilities," where a malicious prompt can flow through the agent…

In this Black Hat Europe 2025 presentation, Dr. Ilia Shumailov argues that traditional software security principles are fundamentally incompatible with modern AI agents. He systematically deconstructs several long-held "security wisdoms" and explains why they fail when applied to AI: Separation of Control and Data Flows: In traditional software, keeping code (control) separate from inputs (data) is a core security tenet. However, AI tasks are inherently data-dependent—the data is often the instruction. While Shumailov’s project, "CaMeL," successfully enforced this separation for tasks that don't rely on data, it highlighted that for most…

"Weaponizing Image Scaling Against Production AI Systems," delivered by Kikimora Morozova: OverviewThe presentation explores a novel attack vector targeting multimodal AI systems (like Google Gemini and Vertex AI). The researchers discovered that attackers can exploit the downscaling algorithms AI platforms use to process media, allowing them to embed invisible or inaudible "prompt injections" that the AI will read and execute. The Core Vulnerability: Lossy TransformationsTo save processing power, AI platforms automatically downscale uploaded images and compress audio. These downscaling algorithms (such as bicubic or nearest-neighbor) are "lossy" and do…

"Exploiting Multi-Agent Systems: How Prompt Injection Turns Collaboration into Compromise" by Jeremy Richards from ServiceNow’s AI Red Team. OverviewThe presentation explores the emerging attack surface of multi-agent AI systems. As AI shifts from single chatbots to complex, multi-agent frameworks capable of autonomous tool use and long-term planning, the "blast radius" of prompt injection attacks significantly expands. Richards argues that the power of a prompt injection is entirely bounded by the implementation—specifically, the privileges and tools granted to the injected agent. Core Concepts: The Multi-Agent ArchitectureRichards explains the standard…

Agentic Edge AI: Threat Architecture, Attack Surfaces & Real-World Risk. Overview In this presentation, Numaan Huq, a Senior Threat Researcher at Trend Micro, explores the rapidly approaching future of Agentic Edge AI—autonomous AI systems deployed on physical devices rather than relying solely on the cloud. Huq outlines what these systems are, how they operate, and critically, the new and complex cybersecurity threats they will introduce over the next 3 to 5 years as they integrate into homes, workplaces, and public infrastructure. What is Agentic Edge AI? Huq begins…