Claude Sonnet 5 with Cursor: strong reasoning, throttled by the harness

We ran Claude Sonnet 5 again for our Agent Security League, this time paired with Cursor instead of Claude Code, on the same dataset. Cursor usually wins these harness match-ups; not this time. Cursor + Sonnet 5 landed at 63.1% FuncPass and 15.6% SecPass, well behind Claude Code + Sonnet 5's 83.2%/19.6%, and the gap traces almost entirely to a Cursor-specific throughput problem for this model, not to worse reasoning.

Key takeaways

• Cursor + Sonnet 5's weakest showing yet. 63.1% FuncPass, 15.6% SecPass, near the bottom of our leaderboard, and well below last week's Claude Code + Sonnet 5 result.
• First time we've seen Claude Code beat Cursor for the same model. Every other model we've tested had Cursor on top. Sonnet 5 flips that.
• It's a throughput story, not a reasoning story. Cursor timed out this model 3x more often than Claude Code, and because it tends to edit late, those timed-out runs often shipped a partial patch instead of a complete one.
• Cheating cuts the wrong way for Cursor here. Moderate by our benchmark's standards, but enough to widen the harness gap instead of closing it.
• The Sonnet 5 vs Fable 5 comparison is its own story. Strip out timeouts and cheating, and a consistent split appears on both harnesses, one model more reliable, the other more security-forward. We break that down in a companion piece.

Introduction

Last week we benchmarked Claude Sonnet 5 with Claude Code: a strong functional performer (83.2% FuncPass), an average security performer (19.6% SecPass), and refreshingly, a model that barely cheated. We flagged at the time that a companion run pairing Sonnet 5 with Cursor was still in progress, and that it was, notably, running far slower with frequent timeouts.

That companion run is done. The short version: Cursor + Sonnet 5 comes in well behind its Claude Code sibling on both metrics, and this time the harness that has won every previous same-model comparison on our board, Cursor, loses. We dug into why, and it is a genuinely different story from the "harness steers reasoning" narrative we told two weeks ago about Fable 5. This one is about time budgets and turn counts, not about which agent asks better security questions.

This post focuses on that Cursor + Sonnet 5 result and the harness reversal it produced. We now also have all four combos on the same dataset (Cursor + Sonnet 5, Claude Code + Sonnet 5, Cursor + Fable 5, Claude Code + Fable 5), which lets us put Sonnet 5 and Fable 5 head-to-head as models rather than harness match-ups, that analysis lives in a separate companion piece.

Sonnet 5 with Cursor: a rough time budget

Its 63.1% FuncPass places it near the bottom of the board, ahead only of Claude Code + Fable 5, and its 15.6% SecPass is lower-middle: above the Opus 4.6 basement (~11%) but below the mid-table cluster (Cursor + Opus 4.7 at 18.4%, Claude Code + Sonnet 5 at 19.6%) and well behind the front-runners (Cursor + Fable 5 at 29%, Cursor + GPT-5.5 at 24%).

Placed next to the Claude Code result, the gap is the largest we've seen between two harnesses running the same model (we discuss this more in the next section):

Cursor hit the timeout limit far more frequently: 65 of 200 tasks (32.5%) timed out on the first pass, compared with 20 of 200 (10%) for Claude Code.

Every one of Claude Code's 20 timeouts still returned a usable (partial) patch. Cursor's did not: 32 of the 65 first-pass timeouts shipped an empty patch, because the harness captures whatever git diff exists at the instant of the hard kill, and Sonnet 5 under Cursor tends to start editing late (a median of ~44% of the way through a near-timeout run, in some cases past 80%).

Because our pipeline retries any instance that produced an empty patch, this triggered all five resume rounds, stretching the total wall-clock for this experiment to roughly 52 hours, versus 14 hours for Claude Code's single pass. From round two onward, nearly every retried instance timed out again, these are chronic non-convergers, not one-off slow days. Net result: 8 of 200 tasks never produced a working patch at all, even after all five retries.

The instances that did eventually ship a partial patch out of a timed-out session performed far below the run's average, 26.9% FuncPass and 9.6% SecPass across the 52 "timeout-with-patch" instances, against 72.5% / 21.5% for the run overall (before anti-cheating and adjustment). Since those 52 instances make up more than a quarter of Cursor + Sonnet 5's total submissions, this subpopulation alone drags the headline numbers down substantially, independent of any cheating or code-quality effect.

Cheating falls roughly in the middle of the range we've observed in our leaderboard: 17 confirmed instances on Cursor (9 workspace leakage, 8 training recall). That's well below the numbers we got with Fable 5 (29 confirmed cases for Cursor + Fable 5 and 38 for Claude Code + Fable 5), but above the notably clean 8 cases for Claude Code + Sonnet 5. Adjusting for cheating, therefore, shifts Cursor's result slightly away from Claude Code rather than toward it. However, the effect is modest compared with the much larger timeout-driven gap discussed above.

Sonnet 5 across harnesses: the first reversal we've seen

We've now run four models, Fable 5, Opus 4.6, Opus 4.7, Opus 4.8, through both Cursor and Claude Code on this dataset. Cursor won on every metric, every time. But this is reversed in Sonnet 5:

This has a straightforward explanation. The same two mechanisms we already examined above while discussing the low scores for Cursor + Sonnet 5 are sufficient to explain the reversal:

1. Longer runtime. Cursor's much longer per-instance turnaround pushed a larger share of submissions into timeouts, resumes, and partial or empty patches.
2. Higher cheating. Cursor's confirmed-cheating count, while unremarkable in absolute terms, runs a bit higher than Claude Code's for this model (17 vs. 8).

While we do not have the means to explain why a model cheated more than another, the longer runtime point is worth more investigation.

Why does Cursor take so much longer for this Sonnet 5 model? We compared the 140 instances that both harnesses completed. We could rule out the two obvious infrastructure suspects: cache-hit ratio was nearly identical (97–98% on both sides), and median per-turn latency was also nearly identical (4.44s Cursor vs. 4.28s Claude Code). What differs is turn count and latency tail: Cursor needs ~45% more agentic turns per instance (median 46 vs. 32) and has a noticeably heavier slow-turn tail (mean latency 1.22x higher, p90 1.42x higher). In short: same model, same cache behavior, same typical response speed, but Cursor's tool-calling loop, as served for this particular model configuration, needs many more round trips to converge, and those round trips occasionally take much longer than usual, which is what blows through the time cap and produces the timeouts, resumes, and empty patches described above.

Two weeks ago, our Fable 5 companion piece argued the harness gap was "mostly patch quality", Cursor steering the model toward more complete security fixes on shared tasks. It's worth being explicit that this is not that story. For Sonnet 5, we did not find Claude Code writing systematically weaker patches on tasks it actually finished; we found Cursor failing to let the model finish on a much larger share of tasks in the first place.

That is a meaningfully different failure mode from the Fable 5 case, and it's also a caution against reading any single harness comparison as a durable "Cursor is the better scaffold" rule. It depends heavily on how a specific model performs, latency- and turn-count-wise, once it's routed through a specific harness's tool-calling loop, and that can vary a lot by model, as this experiment shows.

Conclusion

Cursor + Sonnet 5 is the weakest combo we've published, but the number to remember isn't 63.1% FuncPass, it's the 32.5% first-pass timeout rate behind it. This wasn't a model that reasoned poorly; it was a model that frequently didn't get to finish. Strip away the timed-out and empty-patch instances, and Sonnet 5's underlying quality looks nothing like its headline score, which is exactly why the same model tops the board when Claude Code gives it room to run.

Two broader lessons carry forward. First, harness rankings are not durable: Cursor had won every prior same-model comparison on our board, and Sonnet 5 reversed it purely on throughput, so "which scaffold is better" is really "which scaffold suits this model's turn-count and latency profile." Second, timeouts and cheating are confounds that have to be controlled for before any raw FuncPass/SecPass comparison means much.

That second point is where this experiment gets genuinely interesting. With all four Sonnet 5 and Fable 5 combos now on the same dataset, we can strip out both confounds and compare the two models directly, and a consistent, non-obvious split emerges: one model is more reliable and thorough, the other more security-forward. We break it down across three confound-resistant dimensions in the companion piece: Sonnet 5 vs Fable 5.