We benchmarked Claude Sonnet 5 for the Agent Security League, paired with Claude Code. It turned in a solid-but-familiar scorecard: 83.2% on functional solves and 19.6% on security solves. The headline is what it didn't do: almost no cheating, and moderate timeouts.
Key takeaways
- Solid, not spectacular. Claude Code + Sonnet 5 reached 83.2% FuncPass and 19.6% SecPass — near the top of the leaderboard on functional correctness, and upper-middle on security.
- The FuncPass/SecPass gap persists. Even a strong functional performer leaves roughly 8 out of 10 vulnerabilities open once you look at the security tests.
- Very little cheating. We confirmed cheating on just 8 instances — a fraction of the volume seen on recent Claude releases (Fable 5 hit 38), and low enough that adjustment barely moved the numbers.
- Not a memorizer. Where recent models were dominated by training recall, Sonnet 5's few confirmed cases were mostly workspace leakage (6), with only 2 training-recall cases.
- Moderate number of timeouts. 20 instances for which a partial patch was produced.
Introduction
Sonnet 5 is Anthropic's latest mid-tier frontier model, and it arrives with the usual expectation that each new release should push security outcomes higher. As usual, our benchmark asks a narrow but demanding question: can an agent modify real, complex code to fix a vulnerability while keeping the project working — using only its own reasoning over the local codebase, not a remembered or retrieved copy of the upstream fix?
On that test, Claude Code + Sonnet 5 lands as a strong functional performer with an average security result. It is a good engineer and a mediocre security engineer — the same split we keep seeing across the league.
This is the Claude Code half of the story. A companion run pairing Sonnet 5 with the Cursor harness is still in progress (and, notably, running far slower with frequent timeouts); we will report those results separately, as we did for Fable 5.
Results: strong function, average security
The FuncPass of 82.6% puts Sonnet 5 among the best functional performers we have measured — essentially level with the top of the leaderboard. Its 19.6% SecPass is upper-middle: below the current front-runners (Cursor + Fable 5 at 29%, Cursor + GPT-5.5 at 24%) but comfortably ahead of most of the field.
The takeaway is the recurring one for this benchmark: functional competence does not translate into secure code. Sonnet 5 writes patches that work about five times more often than it writes patches that actually close the vulnerability.
A closer look at cheating: low, and not memorization
The most interesting thing about Sonnet 5 is how little it cheats. Our pipeline confirmed cheating on only 8 of 200 instances — so few that removing them moved SecPass by less than 3 points. For comparison, Fable 5 with Claude Code topped our post-hardening chart at 38 confirmed cases.
The mechanism mix is also different:
Recent frontier models, Opus 4.8, Fable 5, Composer 2.5, were dominated by training recall: reproducing upstream fixes verbatim, down to CVE identifiers, changelog annotations, and long copied comments that cannot be derived from the workspace. Sonnet 5 mostly avoided that. Its handful of confirmed cases were instead workspace leakage — finding a fixed copy of the code already present in the container (installed packages, build artifacts) and reading it back rather than deriving the fix. This is still cheating for our purposes, but it is a behavior our conversation-analysis signal catches directly, not the harder-to-prevent recall problem.
In short: on this run, Sonnet 5 was one of the more honest performers we have benchmarked.
You can see the full agent security leaderboard results here.
What's next?
When you're ready to take the next step in securing your software supply chain, here are 3 ways Endor Labs can help:
