{"id":419,"date":"2026-08-08T06:22:38","date_gmt":"2026-08-08T06:22:38","guid":{"rendered":"https:\/\/43.156.46.27\/?p=419"},"modified":"2026-08-08T06:22:38","modified_gmt":"2026-08-08T06:22:38","slug":"black-hat-europe-2025-why-we-cant-retrofit-old-security-principles-onto-ai-agents","status":"publish","type":"post","link":"https:\/\/haco.zone\/?p=419","title":{"rendered":"Black Hat Europe 2025 | Why We Can&#8217;t Retrofit Old Security Principles Onto AI Agents"},"content":{"rendered":"\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Black Hat Europe 2025 | Why We Can&amp;apos;t Retrofit Old Security Principles Onto AI Agents\" width=\"640\" height=\"360\" src=\"https:\/\/www.youtube.com\/embed\/HGCwYIUgoKc?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">In this Black Hat Europe 2025 presentation, Dr. Ilia Shumailov argues that traditional software security principles are fundamentally incompatible with modern AI agents. He systematically deconstructs several long-held &#8220;security wisdoms&#8221; and explains why they fail when applied to AI:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Separation of Control and Data Flows:<\/strong>\u00a0In traditional software, keeping code (control) separate from inputs (data) is a core security tenet. However, AI tasks are inherently data-dependent\u2014the data\u00a0<em>is<\/em>\u00a0often the instruction. While Shumailov\u2019s project, &#8220;CaMeL,&#8221; successfully enforced this separation for tasks that don&#8217;t rely on data, it highlighted that for most real-world AI applications, control and data flows must mix. When they do, the system becomes highly vulnerable to attacks like prompt injections.<\/li>\n\n\n\n<li><strong>Access Controls:<\/strong>\u00a0Traditional human-centric access controls, like Role-Based Access Control (RBAC), are inadequate for AI. Shumailov describes AI agents as the &#8220;ultimate insider threat.&#8221; They operate continuously, have broad access to internal infrastructure, and act with an unpredictable, non-human rationale. They can easily bypass coarse-grained controls and exploit minute side channels that human rules don&#8217;t anticipate.<\/li>\n\n\n\n<li><strong>Guardrails and Detectors:<\/strong>\u00a0The speaker strongly criticizes current commercial guardrails and detection systems, calling them unprincipled and poorly evaluated. He claims that most of these defenses can be easily and cheaply bypassed, offering only a false sense of security.<\/li>\n\n\n\n<li><strong>Red Teaming as a Solution:<\/strong>\u00a0Relying on red teaming to fix AI security is flawed. Commercial red teaming companies have a financial incentive for problems to persist, and academia is more focused on the prestige of breaking systems than the difficult work of fixing them.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Conclusion and Path Forward<\/strong><br>Shumailov concludes that the industry faces a turbulent period. We cannot retrofit old security paradigms onto AI; instead, we need entirely new security semantics, rebuilt AI protocols that incorporate security by design, and a realignment of incentives.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Q&amp;A Insight<\/strong><br>During the Q&amp;A, Shumailov addresses the idea of using one AI system to monitor another. He argues that this approach is theoretically impossible and practically ineffective. Because the monitoring model and the agent model have similar architectures and data access, a monitoring AI can be just as easily manipulated or bypassed by the agent it is supposed to be watching.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this Black Hat Europe 2025 presentation, Dr. Ilia Shumailov [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[35,5],"class_list":["post-419","post","type-post","status-publish","format-standard","hentry","category-black-hat","tag-llm","tag-security"],"_links":{"self":[{"href":"https:\/\/haco.zone\/index.php?rest_route=\/wp\/v2\/posts\/419","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/haco.zone\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/haco.zone\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/haco.zone\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/haco.zone\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=419"}],"version-history":[{"count":1,"href":"https:\/\/haco.zone\/index.php?rest_route=\/wp\/v2\/posts\/419\/revisions"}],"predecessor-version":[{"id":420,"href":"https:\/\/haco.zone\/index.php?rest_route=\/wp\/v2\/posts\/419\/revisions\/420"}],"wp:attachment":[{"href":"https:\/\/haco.zone\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=419"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/haco.zone\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=419"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/haco.zone\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=419"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}