**On November 10, 2026, Wiz Research published how it rebuilt Cyber Model Arena v3.9: 26 models, four harnesses, 59 agents, 327 challenges, three attempts each, and 1,000 seconds per attempt, grading with code—not LLM judges— on isolated targets. The benchmark includes stacks readers actually run—Claude Code, Codex, ADK React, and Antigravity paired with frontier models. If you operate coding agents with shell and network tools, these pass@1 numbers are a calibrated warning about what model + harness can automate on real exploit tasks, not trivia.
TL;DR
| Question | Answer |
|---|---|
| What is being measured? | Checkable exploit effects on isolated targets from instruction + harness + tools, not multiple-choice knowledge |
| Top-line scale (Nov 10) | 77.2% pass@1 for ADK React + Gemini 4 Argon; Claude Code + Claude Opus 5 at 71.2% per Wiz appendix |
| Hardest category | Code vulnerabilities averaged 38.2% pass@1 across agents; API security averaged 78.0% |
| Key design choice | No internet, no answer leakage, deterministic graders |
| Harness effect | ~5.1 point mean absolute gap when the same model runs on different harnesses |
| Beam relevance? | Local monitoring of what your harness does—not a substitute for cloud-side posture tools |
Why Wiz built it
Wiz frames a gap between public cyber-agent benchmarks and defender questions. Multiple-choice tests measure model knowledge, and LLM-judged outputs measure agreement between models, not whether an attacker achieved an effect on a target.
Wiz’s stated objective:
Given only the task instruction, a general-purpose agent scaffold, a 1,000-second budget, and no external knowledge sources, does the agent produce a checkable effect on a real target?
That aligns with field reality Wiz cites from its 2026 State of AI in the Cloud Report: 57% of organizations reported self-hosted AI agent technology, and MCP servers appeared in 80% of observed cloud environments. Agents are already in production paths attackers study.
Methodology in plain language
Fix objective before tasks
Wiz insists on settling why (actionable buyer/defender metric), what (model + harness + tools under evaluation), and how (single bottom-line metric with constraints) before writing challenges. Competing metrics without reduction become debates about scoring, not security decisions.
Four equal-weight categories (327 tasks)
Wiz built tasks from wild sources, not LLM-generated labels:
| Category | Tasks (v3.9) | What it tests |
|---|---|---|
| Code vulnerabilities | 221 | Known bugs in stripped OSS or live 1-day CVE exploitation |
| API security | 54 | Black-box APIs without source |
| Web security | 29 | Real PHP apps with client source visible |
| Cloud security | 23 | Misconfigurations across AWS, Azure, GCP, Kubernetes-style setups |
Category averages diverge sharply. Wiz reports 78.0% pass@1 on API security versus 38.2% on code vulnerabilities averaged across agents. A single overall score without category balance oversells generalization.
Grade effects, not opinions
Wiz rejects LLM-as-judge as the primary grader: judges drift between runs, cost money, introduce bias (including position bias), and measure whether a model thinks the agent succeeded.
Instead, each challenge leaves a checkable artifact—file state, structured output, or service effect—and a deterministic program scores it after the agent exits.
Anti-cheat via isolation
Agents shortcut when they can. Wiz cites work showing web-search access moved solve rates by up to 12 points in another benchmark. Cyber Model Arena isolates agents from the internet and challenge servers, blocking lookup of walkthroughs or CVE pages mid-run. That does not prove training-data memorization never helps—it proves runtime cheating is harder.
Attempts and cost
Real attackers retry. Wiz grants three attempts per challenge; pass@2 and pass@3 rise for many stacks. It also plots success vs dollars and minutes per solve, highlighting agents off the Pareto frontier—for example, Claude Code + GPT-5.6 Sol at 64.2% pass@1 but $8.29 and 27.2 minutes per solve versus Codex + GPT-6 Luna at 64.5% with $0.05 and 6.3 minutes in Wiz’s table.
What the November 10 leaderboard shows
Wiz’s appendix orders 59 agents by pass@1. Illustrative rows readers may recognize:
- ADK React + Gemini 4 Argon — 77.2% pass@1, $0.77 / 4.3 min per solve @1
- Claude Code + Claude Opus 5 — 71.2% pass@1
- Codex + GPT-6 Sol — 71.2% pass@1
- Claude Code + GPT-6 Luna — 60.5% pass@1 ($0.09 / 21.1 min @1)
- Claude Code + Grok 4.6 — 45.5% pass@1
Low-end examples include ADK React + MiMo V2.6 Pro at 39.7% and Claude Code + MiMo V2.6 Pro at 39.0%. Grok on Claude Code scores 45.5% overall but Wiz notes an agent at 9.7% on code vulnerabilities while still 71.6% on API tasks—category choice swings headlines.
Harness pairing: Wiz measured 5.1 points mean absolute pass@1 gap when the same model runs on Claude Code versus ADK React for 22 models. Conclusion in the post: evaluate the system, not the weights file alone—either match harness to model vendor or fix one harness and compare models inside it.
Reading the category spread as a defender
Wiz’s category averages are not cosmetic—they predict which automations succeed first against your stack:
- API security (78.0% average pass@1) — agents already excel at black-box HTTP abuse. If your coding agent can reach internal APIs with stolen cookies or mis-issued tokens, arena scores suggest automation is mature, not experimental—consistent with ARTEX-style agentic intrusion reporting.
- Web security (64.6%) — client-visible PHP sources plus tool loops still yield high success; relevant when agents clone vulnerable tutorial apps into corporate networks.
- Cloud misconfigurations (56.5%) — still majority-success averaged across agents; pairs with Qubole trusted-role abuse themes where cloud IAM is the exploit surface.
- Code vulnerabilities (38.2%) — hardest category, but top agents still exceed 65% on code in Wiz’s examples. Do not treat “low average” as “safe against determined attackers with budget.”
For local developer risk, the lesson is narrower: the same harness that scores well on API tasks is the one developers run with shell, git, and cloud CLIs. Arena isolation removes internet cheating; your workstation does not.
Limitations Wiz admits
The November 10 post lists honest bounds:
- Only four simple harnesses—specialized offensive frameworks might score higher.
- String equality on code-vulnerability root causes—semantically correct analysis that misses the expected symbol scores zero.
- Public web tasks still risk training contamination; isolation stops fetching answers, not remembering them.
- 1,000-second cap may not match unconstrained attacker patience; future work may cap cost instead of time.
- Confidence intervals not yet on the public site.
Readers evaluating enterprise buys should read Wiz’s downloadable methodology rather than headline pass@1 alone.
What this means if you run AI coding agents
These scores describe offensive automation potential on isolated CTF-like targets, not your IDE’s daily refactor workload. They still matter locally:
-
Tool-rich harnesses are weapons-grade scaffolds. Claude Code and Codex entries in the mid-60s–71% pass@1 range are not “sometimes helpful autocomplete.” They are general attack loops when pointed at reachable services and secrets—consistent with Chainalysis “offensive agentic techniques” reporting and Wiz’s agentic attack fingerprints.
-
Category gaps mirror how teams under-test agents. High API scores and low code-vulnerability scores suggest many stacks excel at HTTP abuse while struggling on source-heavy flaw finding—the inverse of how some teams use agents only for app code, not dependency audits. Pair arena insights with dependency CVE posts and
beam cve checkworkflows. -
Harness choice is policy. If your org standardizes on one CLI, Wiz’s 5.1-point harness gap is an argument for consistent hook placement and tool allowlists per harness—not chasing the highest arena model name each week.
-
Isolation in the benchmark ≠ isolation on your laptop. Arena runs block internet cheating. Your developer machine often does not. Assume agents can curl, read
.env, and probe localhost unless hooks deny it. -
Red team ≠ production monitoring. Benchmarks answer capability. Bay’s November 10 launch and enterprise posture tools answer inventory and deny at scale. Individual contributors still need local evidence when something goes wrong.
Practical policy responses (no hype)
Wiz’s post is not a shopping list for new SaaS. Concrete moves for teams already running Claude Code or Codex:
- Shrink default tool allowlists — remove cloud CLIs from profiles used for documentation-only tasks.
- Separate harnesses — red-team profiles on disposable VMs; daily coding on enforced workspace policy with deny rules for
curlexfil patterns and destructiverm. - Log review after incidents — arena scores justify post-facto investigation even when benchmarks were never run internally.
- Do not treat pass@1 as “model safety.” Alignment evaluations like Anthropic’s incident studies cover different failure modes than live exploit effects.
For evaluation methodology comparisons, see our buyer’s guide to agent security platforms and Anthropic alignment incidents.
How Beam handles this
Wiz Cyber Model Arena measures offensive success rates in a controlled lab. Beam does not benchmark models. It is local-first observability for Claude Code and Codex on your machine: it records tool calls and commands, runs heuristic scans on skills and MCP configs, and supports hook-based PreToolUse deny when workspace policy is enforce.
Where Beam fits after reading Wiz’s numbers:
- After a red-team agent run, Beam’s on-device event history helps you answer what tools fired and whether destructive or credential-shaped actions occurred—complementing a pass@1 score with auditable traces (monitoring basics).
- Before you run unfamiliar skills or MCP entries, Beam’s eleven-pattern scan catches known-bad shapes (curl-pipe-bash, secret literals)—not semantic malware analysis.
- Beam does not block network egress by default, does not replace Wiz-style cloud graphing, and fails open if cached deny policy is missing. It is the developer laptop slice of defense, not an enterprise arena.
See local-first collection and how Beam watches.
Frequently asked questions
What is Wiz Cyber Model Arena v3.9?
Wiz’s isolated offensive-agent benchmark with 327 tasks, 59 agent stacks, and code-based grading as described in its November 10, 2026 blog post.
Does Claude Code score high?
Claude Code + Claude Opus 5 shows 71.2% pass@1 in Wiz’s appendix—top tier but not the single highest entry.
Should I pick models by this leaderboard?
Use it to understand automation potential, not to select a code-review assistant. Match model choice to task and harness policy.
Why avoid LLM judges?
Wiz argues judges are biased, noisy, expensive, and measure opinion, not effects—hence code graders.
What about MCP in production clouds?
Wiz cites MCP prevalence in cloud environments in its State of AI in the Cloud Report; arena tasks are not MCP-specific, but the tooling surface is the same class of risk Beam scans locally.
Can Beam improve my arena score?
No. Beam is defensive observability, not an offensive harness tuner.
Related reading
- AI agent security platforms: how to evaluate
- Anthropic alignment assessment and real incidents
- Wiz agentic attack fingerprints
- Chainalysis offensive agentic techniques
- Bay AI agent security runtime hooks launch
- Black October: AI developer platforms and stolen keys
- Wiz Green Agent ASPM handoffs to coding agents
Primary source: Wiz — How to Evaluate AI Agent Cybersecurity Capabilities (November 10, 2026). Cyber Model Arena results: Wiz Cyber Model Arena.
Task counts, model names, and pass@1 figures match Wiz’s November 10, 2026 publication.