RECON 2026 – Deobfuscation in the Age of Agentic Reverse Engineering

RECON 2026 – Deobfuscation in the Age of Agentic Reverse Engineering, presented by Tim and Nicolò from the Recon Conference:

Overview & Motivation

  • Context: In 2022, the speakers argued that obfuscation was outpacing deobfuscation due to inter-procedural complexity, custom tooling requirements, and a lack of scalable automation [00:39].
  • The Shift: Autonomous AI agents—LLMs running inside execution loops with direct tool access—are fundamentally altering the economics and speed of reverse engineering [01:21].

The Agentic Setup

  • Environment: Frontier LLMs (e.g., Claude Code, Codex) run inside sandboxed Docker containers [02:32].
  • Toolchain Integration: The agents are granted programmatic API access to reverse engineering platforms and libraries, including Binary Ninja, Unicorn (CPU emulation), Miasm (symbolic execution), and Z3/MSon (Mixed Boolean-Arithmetic simplification) [02:40].

Case Studies

1. Virtual Machine (VM)-Based Obfuscation

  • Simple Function-Level VMs: Given a high-level goal, the agent can autonomously discover the VM dispatch architecture, extract handler semantics, write a custom disassembler, and lift bytecode back to high-level code in minutes (“one-shotted”) [05:54].
  • Commercial Protections (VMProtect, Themida):
    • Features like handler duplication (10k–50k handlers) and rolling bytecode decryption cause unguided agents to fail [06:13].
    • When given a structured trace-based workflow (Unicorn trace →→ symbolic execution →→ handler recovery →→ disassembler generation), the agent resolved complex protections in 1–2 hours—a task that typically takes weeks manually [09:48].
  • Kernel Anticheats & Game DRMs:
    • Successfully identified shared VM interpreters across over 1,000 entry points in a kernel anticheat, devirtualizing code in parallel [12:18].
    • On older game DRMs (5–7 years old), the pipeline recovered license integrity cryptographic logic [14:20].
    • Modern DRMs resisted analysis due to heavy anti-emulation loops and requirements for live dynamic runtime state [15:38].

2. Control Flow Flattening (CFF)

  • Emotet Banking Malware: The agent authored custom Binary Ninja workflows targeting Medium-Level Intermediate Representation (MLIL). It located dispatchers, resolved block successors, and patched direct control flows across all flattened functions [19:28].
  • FinFisher: Reusing the generic Emotet workflow failed due to sample-specific Mixed Boolean-Arithmetic (MBA) opaque predicates, requiring custom adaptations [20:38].

3. Global Call Graph Obfuscation

  • Obfuscators hide direct function calls behind indirect lookups using global tables and MBA indices [22:04].
  • Unguided agents pursued dead-end brute force and heuristic attempts [23:44]. With targeted human direction (constant propagation and MBA simplification via Z3), the agent resolved indirect targets back to direct calls and reconstructed clean function boundaries [24:11].

Countermeasures: Anti-Agentic Obfuscation

The speakers outlined four design principles for defenders to build protections specifically resistant to agentic analysis [25:51]:

  1. Search Budget Traps: Bloating state spaces, introducing spaghetti logic, and adding costly oracle checks to exhaust agent token limits and execution budgets.
  2. Global Semantics & Runtime Binding: Tying critical program execution to inputs only available at live execution time (e.g., cross-thread interactions), frustrating purely static/emulated analysis.
  3. Tool Friction: Designing targeted anti-debugging and anti-emulation tricks (e.g., subtle hardware modeling flaws in QEMU) to break the underlying tools agents rely on [28:28].
  4. Narrative Poisoning: Planting false-flag strings, deceptive comments, or prompt injection sequences within binary metadata to mislead LLM reasoning.

Leave a Reply

Your email address will not be published. Required fields are marked *