The AI security competition once again proves a point: the winners are the teams that build the best systems, not the teams with the most powerful models. Trend Micro’s TrendAI team recently announced that its agentic exploit-remediation engine, codenamed AESIR, took first place in UC Berkeley’s CyberGym benchmark with a score of 97%, leading second-place Sangfor AI (93.2%) by nearly 4 percentage points, and significantly surpassing OpenAI’s GPT-5.6 Sol (84.5%) and Anthropic’s Claude Mythos 5 (83.8%).
What is CyberGym?
CyberGym is an AI safety benchmark created by a research team at UC Berkeley, covering 1,507 real-world vulnerabilities across 188 large open-source software projects. The test tasks are not as simple as just crashing a program. For each vulnerability, the system must produce a “differential crash-triggering input” (differential crash): a specific byte sequence or file that must crash the old (vulnerable) program while the patched version runs normally. The system cannot access the patch or the fixed version when solving the task.
This is far above the standard of typical fuzzing, as it requires the system to prove that its inputs actually triggered the specific vulnerability that developers fixed, rather than any random instability. The vulnerability types cover heap overflow, use-after-free, integer overflow, uninitialized memory reads, and stack corruption—these are precisely the root causes behind the large number of CVE reports every year.
The test projects include widely used industry software such as FFmpeg, Wireshark, and Ghostscript, as well as various codec libraries and smart card middleware.
CyberGym Ranking (August 2026)
AESIR leads by a wide margin with a score of 97%. Here are the main competitors on the leaderboard:
- 1st place：TrendAI AESIR，97.0%(Trend Micro)
- 2nd place：Sangfor AI，93.2%(Sangfor)
- 3rd place：Whitzard (Bai Ze),91.2%(Fudan University)
- 4th place：MDASH，91.0%（Microsoft）
- 5th place：Wiz Atlas，90.9%（Wiz）
- 9th place：GPT-5.6 Sol，84.5%（OpenAI）
- 12th place：Claude Mythos 5，83.8%（Anthropic）
Looking at the leaderboard, all the competitors near the top (Sangfor, Fudan University, Microsoft, Wiz) are using the same frontier AI models. The difference comes entirely from the engineering around the models—this is a competition of “systems,” not “models.”
AESIR’s Secret Weapon: The Staged Pipeline
The TrendAI team explained in detail the core design of AESIR in their technical blog. The system adopts a graduated pipeline with escalating costs, exhausting all low-cost deterministic methods before calling on AI inference.
The first phase is classic fuzzing combined with seed mutation, using proven exploit structures extracted from the known vulnerability ontology accumulated through TrendAI’s more than 20 years of vulnerability research (supported by Zero Day Initiative, with over 12,500 records) as a starting point. This phase can solve approximately 30% of the tasks without any AI involvement, taking less than 2 minutes per task at zero cost.
The system only escalates to AI-driven agents when previous stages fail. AESIR uses a total of seven models from four providers (Anthropic, Google, OpenAI, DeepSeek), with each model playing a different role: Claude Opus handles deep analysis tasks requiring long-range reasoning (about 31% of verified tasks), Gemini Pro performs unexpectedly well at binary format construction, GPT excels at protocol-specific tasks, and DeepSeek contributes on specific code patterns that appear extensively in its training data.
CyberGym’s paper also found that the overlap in success rates between different agent frameworks is less than 50%. In other words, different models solve different subsets of tasks, and a single-model system would miss nearly half of the solvable problems.
Adversarial Quality Assurance: The False Positive Killer
AESIR’s third key design is a “proposer-adversary-adjudicator” three-layer architecture. The system does not let the same model self-verify its own findings, because models are far too willing to agree with their own outputs. Before any vulnerability hypothesis is released, it must pass through another vendor’s model that actively attempts to refute it.
For example, in round N, Claude Opus is the proposer (putting forward a vulnerability hypothesis), GPT-5.6 Luna is the opponent (trying to refute it), and Gemini 3.1 Pro is the adjudicator. In round N+1, the roles rotate: GPT becomes the proposer, Gemini becomes the opponent, and DeepSeek becomes the adjudicator. No model knows its role assignment in advance, and no single model can decide the outcome on its own.
The TrendAI team emphasized: “The goal of adversarial QA is to kill findings, not confirm them. A finding that survives structural rebuttal by different models is the only one worth reporting.” This mechanism solves the most troublesome problem in AI security tools: false positives. The article cites a 2024 study by Yangruibo Ding et al., which found that GPT-4 paired with chain-of-thought reasoning achieved only 12.94% pairwise accuracy in vulnerability classification—even lower than the 22.7% random guessing baseline. The problem is that models are too easily led astray. Ask them to find vulnerabilities, and they’ll produce a pile of patterns that look like vulnerabilities but are actually protected by upstream validation or don’t exist on reachable paths.
When AI Isn’t the Answer: 30% of Tasks Don’t Need LLMs at All
One counterintuitive statistic: of the roughly 1,460 exploit cases AESIR verified, nearly a third required no AI at all. Classic fuzzing outperformed AI reasoning on about a quarter of the tasks. A 60-second fuzzer with good initial inputs from related vulnerabilities solved tasks that took AI agents 18 minutes and $6 in API costs. The zero-LLM tasks were also 15 times faster.
TrendAI’s conclusion is clear: “Direct the task to the right tool. The most expensive tool is rarely the right one.”
Economic benefit: $6,000 in API fees vs $4.88 million in data breach costs
From an economic perspective, the 97% score behind it represents approximately 1,460 independently verified differential crashes across 188 open-source projects. The total cost was roughly $6,000 in AI API fees plus 235 CPU hours. For comparison, according to IBM’s 2024 report, the average cost of a single enterprise data breach is $4.88 million. Moreover, once the knowledge base is built, the marginal cost of scanning the next project approaches zero.
Lessons for the information security engineering team
In the article, the TrendAI team made several suggestions with broad relevance to the industry:
- First, establish the knowledge layer.The compound returns from continuously accumulating institutional knowledge far outweigh better prompts or better foundation models. Every task without a knowledge layer is a task that doesn’t make the next task easier.
- Before better instructions, first invest in successful examples.Verified exploit cases obtained from related tasks, when mutated and applied to new tasks, outperform hours of AI reasoning.
- Establish adversarial QA before it’s needed.False positive issues in AI security tools accumulate far faster than teams can clean them up.
- Using a multi-model architecture from day one.Different models solve different subsets of problems; a single model leaves structural capability gaps.
- Treat the stepped pipeline as a cost control mechanism.Let cheap and fast methods go first, and only upgrade to expensive AI inference after they fail—otherwise, system costs end up three to four times higher.
Summary
Trend Micro’s TrendAI AESIR took first place in the CyberGym benchmark with 97%, which is an achievement in itself, but what’s more significant is the industry trend it reveals: the AI security race is shifting from “who has the best model” to “who has built the best system.” The 30% zero-LLM solutions, multi-model pipelines, continuously accumulating knowledge ontologies, and adversarial quality assurance—the sum of these engineering decisions is the real reason AESIR leads OpenAI and Anthropic by more than 12 percentage points.
CyberGym is just one of roughly 175 cybersecurity AI benchmarks currently available, but with 1,507 real vulnerabilities, 188 real projects, and byte-level differential verification, it stands as the most comprehensive binary vulnerability reproduction testbed. AESIR’s results validate a conclusion that is gaining wide acceptance: models are becoming commoditized, but the knowledge layer, pipelines, and engineering discipline are not.
Source: KOCPC Chinese