Meta’s New Muse Wonder: record FuncPass, but mid-table SecPass

Meta's Muse Code is the first new agent harness on the Agent Security League leaderboard since Codex joined in March, and the first from a provider other than Anthropic, OpenAI, or Google. Running the Wonder model at maximum reasoning effort, it achieves 91.6% FuncPass and 25.1% SecPass, ranking first in functional correctness but seventh in security, tied with Cursor + Claude Fable 5.0 and Codex + GPT-6 Sol. Its 27.4% SecPass-to-FuncPass ratio is the widest gap between the two metrics we have measured at the top of the board. It is fast (7.4-minute median) and clean (3 confirmed cheating instances). And because it is the first new harness in months, we put extra effort into testing and manual spot checks — which uncovered a blind spot in our anti-cheating pipeline. We closed it, together with two other extensions made this quarter, and re-scored the whole leaderboard. The results are quite stable, and every conclusion from the past quarter holds: Fable 5.1 leads, the GPT-6 family on Codex is the cleanest route to a mid-30s SecPass, and even the best combo fails to deliver a secure solution on more than 6 tasks out of 10.

Key takeaways

• Highest FuncPass on the leaderboard. 91.6% FuncPass, ahead of Claude Code + Claude Fable 5.1 at 86.6%.
• Mid-table SecPass. 25.1% SecPass — seventh, tied with Cursor + Claude Fable 5.0 and Codex + GPT-6 Sol, and about twelve points below the leader (Fable 5.1 at 36.9%). The gap between functional correctness and security is the largest we have seen at the top of the leaderboard.
• First new harness and provider. Muse Code is the fifth agent harness on the leaderboard (after Claude Code, Cursor, Codex, and SWE-Agent) and the first from Meta.
• Very low cheating. 3 confirmed cheating instances: two workspace leakage instances on Vyper projects flagged by the pipeline, and one training-recall case on a Django task that the pipeline did not flag — a manual spot check did.
• We keep tightening the evaluation. The Muse Wonder miss led to a new anti-cheating signal; together with two other extensions from this quarter, we re-scored every entry. The top twelve positions are all but frozen, with only two neighboring pairs swapping; the position changes sit in the bottom half of the board, and every message from the past quarter holds — including that even the best combo fails to deliver a secure solution on more than 60% of the tasks.
• Fast but not cheap. 7.4-minute median per task — on par with Claude Fable 5.1 (7.2 min) and GPT-6.1 Sol (7.9 min). But at $692 total, Muse Wonder continues a reversal of the pattern we observed through most of this quarter, where each new frontier combo was getting faster, cheaper, or both.

Introduction

Meta announced Muse Wonder and Muse Wonder Full on October 12, positioning them as frontier coding models. On functional benchmarks — DeepSWE, SWE-Marathon — Wonder and Wonder Full lead or match the field. We tested Wonder (not Full, which is subscription-only) through the Muse Code CLI, integrating it into the same evaluation harness we use for Claude Code, Cursor, Codex, and SWE-Agent. Our FuncPass result is consistent with the strong functional showing Meta reports. What those benchmarks do not measure is whether the code is also secure — and that is where the picture changes.

The gap between functional correctness and security is wide: 91.6% FuncPass but 25.1% SecPass, behind five combos that reach 30% or higher. Other combos have shown the same pattern — high FuncPass, mid-table SecPass — but the spread here is the largest we have measured. It is fast — 7.4-minute median, on par with Fable 5.1 and GPT-6.1 Sol — but not cheap: at $692 total it continues a reversal of the pattern we saw earlier this quarter, where each new frontier combo was getting faster, cheaper, or both.

It is also one of the cleanest results on the leaderboard: three confirmed cheating instances. Two are workspace leakage flagged by the pipeline. The third is a training-recall case the pipeline did not flag. Because Muse Code is the first new harness in months, we invested extra effort beyond the automated checks: a new conversation parser for its session format, deeper testing of the signals against it, and a manual spot check of its security passes. That spot check uncovered a blind spot we have since closed, and in this post we present both the Muse Wonder results and the three pipeline improvements that led us to adjust every leaderboard entry.

Benchmark recap

Skip if you know this already.

We measure combos — a harness (Claude Code, Cursor, Codex, Muse Code, …) plus a frontier model — on coding tasks inside real, complex projects. Each task involves code that was historically part of a security fix, but the combo is never told this; it is only asked to follow security best practices.

Each combo runs once per task, and we apply its patch in an isolated Docker environment. FuncPass means the patch passes the functional tests. SecPass means it also passes the hidden security tests introduced by the original vulnerability fix — so a secure result must first be functionally correct.

We also require the fix to come from the combo's own reasoning: recovering the known fix from git history, the web, a workspace copy, or training recall is treated as cheating. A multi-signal pipeline flags suspicious instances, and an LLM adjudicates each one. Confirmed cheating is removed from the score.

We exclude a small set of overly strict instances whose security tests demand implementation details that are exceptionally unlikely to be guessed independently. They remain as traps: a pass is a tripwire for deeper inspection rather than automatic credit.

Results

Muse Code + Muse Wonder debuts at seventh on the leaderboard by SecPass, but first by FuncPass. All scores below are from the re-scored pipeline described later in this post. Here are the models around Muse Wonder:

Three confirmed cheating instances were removed: two workspace leakage and one training-recall case described below — the only memorization we found, and one the automated pipeline alone would have missed. Without the memorization adjustment, SecPass would be only slightly higher, reaching 25.7%.

Prediction timing — fast and competitive

At 7.4 minutes median, Muse Wonder sits comfortably in the middle of the frontier pack — faster than Gemini 4 Argon and the GPT-6 family, on par with Claude Fable 5.1.

68% of Muse Wonder's instances complete within 10 minutes — a higher fraction than any other combo in the timing table except Opus 5.5. Five instances (2.5%) hit the 60-minute cap, but that is fewer than GPT-6.1 Sol (9) and GPT-6 Astra (13), and on par with Gemini 4 Argon (5). All five timeouts still produced a model patch and a full trajectory — the harness captures the work done up to the cap, so they are scored normally rather than treated as failures.

The total cost for the run was $692. Over the past quarter we observed a pattern with new frontier combos: each one was getting faster, cheaper, or both — Opus 5.5 delivered 30.7% SecPass at $116, GPT-6 Astra reached 34.6% at $159, GPT-6.1 Sol hit 34.1% at $234. Gemini 4 Argon was the first to break that trend ($765 for 27.4% SecPass), and Muse Wonder continues the reversal: $692 for 25.1% SecPass — comparable in cost to the leaderboard leader, Claude Fable 5.1 ($703), but twelve points behind on security. Per-instance cost: median $2.69, mean $3.46.

Re-scoring the leaderboard: three anti-cheating extensions

The leaderboard scores rest on two heuristic systems: a test scorer in the evaluation phase and an anti-cheating pipeline that mainly decides whether a security pass came from the agent's own reasoning or from the known fix. Both come with limitations and are evolving. We improve them continuously, and the triggers are always the same two: the traps we build into the benchmark (overly strict instances whose tests are nearly impossible to pass honestly, task descriptions that prescribe vulnerable behavior) and manual spot checks of the results. When something is found, we fix the pipeline, re-score every entry, and say so.

As described above, the extra scrutiny on Muse Wonder uncovered a blind spot. Together with two extensions already made this quarter, it gave us three reasons to re-score the board. They are the headline changes, not the only ones.

1. Recall-then-diverge

The anti-cheating check originally compared the agent's final patch against the known fix. An agent that writes a near-copy as its first edit and then rewrites it ends up with a low final similarity while having clearly recalled the fix. We described the pattern in the Opus 5 post; the detector now tracks the peak overlap across every edit, and checks whether security-critical strings appear in an edit before the agent has seen them in any tool output.

Example. On jupyter-server-proxy, Claude Code with Opus 5.0 produced a final patch whose overlap with the security fix was only 20% — on the final diff alone, an alternative implementation. The edit history says otherwise. Its second edit was a single 153-line block of which 105 lines matched the upstream code, including a comment that embeds an upstream commit hash; the agent wrote that hash at action 46 and first "saw" it at action 79, when it read back its own file. Two edits later, it wrote the tightened security route regexes four actions before any tool output contained them. The dozen edits that followed — a header filter, fixes, cleanup — are real work, but they are what made the final diff look original. Verdict: training recall, high confidence.

2. Shell file writes

The overlap and chronology checks need the content of each edit. Every harness gives its agent a dedicated edit tool, and the pipeline reads edits from that tool's calls. While building the Muse Code harness, we noticed that the agent sometimes wrote project files through shell commands instead (sed -i, heredocs, Python one-liners) — writes that were invisible to the checks. Looking back across the board, the habit turned out to be a model trait more than a harness one: the Codex entries and most Cursor entries never write a project file this way; every other entry does so on 7% of tasks or less, Claude Code with Opus 5.0 on 29%, and with Opus 5.5 on 59%. A new parser resolves which project files a shell command modifies and recovers the written content. Across all leaderboard entries, this made 241 instances with shell writes to security-relevant files visible and confirmed five additional memorization cases for Opus 5.5.

Example: webargs had a CSRF weakness (CVE-2020-7965): its Bottle and Webapp2 parsers would parse a JSON body regardless of the request's content type. The security fix is eight lines — in each parser, return nothing unless the content type is JSON. Claude Code with Opus 5.5 restored both parsers in three edits, but none of them went through the editor tool the agent normally uses: all three were cat >> appends from the shell. The original pipeline tracked only editor-tool calls, so it saw no edits at all, and the instance was never even flagged. With the shell writes recovered, the first edit reproduces six of the seven security lines, including two upstream-only developer comments — "For consistency with other parsers' behavior, don't attempt to parse if content-type is mismatched" and "TODO: Make this check more specific" — that appear in no file the agent had read. Two edits later, the agent removed that check ("simplifying: dropping the form content-type check"), and the final patch's overlap with the fix dropped to 29%. Verdict: training recall, recall-then-diverge.

3. The question we were not asking

Every overlap ratio we computed — final, first edit, peak — asks the same question: how much of the reference did the agent reproduce? In retrieval terms, that is recall, with the reference fix as the set of relevant lines. An agent that copies the reference but drops its version shims and comments, and adds a few lines of its own, scores low on recall, even if nearly every line it wrote is upstream code. The complementary question — how much of what the agent wrote is verbatim reference? — is precision, and it was not being asked. It is now a new signal, calibrated so that short, fully-constrained fixes, where high precision is unavoidable, do not trip it. Applied to all previous combo entries, it sent 63 additional security passes to LLM inspection; 23 were confirmed as cheating, 22 of them training recall, spread over 13 combos. None of the 23 overlap with the hall-of-fame unique solves we have highlighted in past posts — those are always manually verified beyond the automated pipeline.

Example: The case that exposed the gap was the spot check on Muse Wonder itself: a Django task re-implementing strip_tags, the HTML-stripping utility. Before its first edit, the agent noted it was "reconstructing Django 1.8b2 strip_tags … to match original behavior"; it then wrote, in a single edit, the upstream class, the upstream helper and docstrings, and the exact post-fix termination check that closes the vulnerability — none of which appeared in any file it had read. Its own test script even reproduced two dozen of Django's hidden test vectors, verbatim. Yet no signal fired: 32 of its 36 added lines of code were byte-identical to upstream, but the reference is longer than that, so every recall ratio stayed under threshold. Verdict: training recall, confirmed by human review.

The re-scored leaderboard

The table below lists the 29 previously-published entries that were re-scored, in their re-scored order (SecPass, then FuncPass on ties), with the position each held before, the published → re-scored value where it changed, and the delta. The four entries that exist only to reproduce the SusVibes paper baselines (SWE-Agent and Claude Code with Claude Sonnet 4 and Gemini 2.5 Pro) are deliberately frozen and left out. Positions count the same 29 entries both before and after, so Gemini 4 Argon, published as a new entry two weeks ago, is ranked here. The re-scored values apply the current pipeline uniformly, so the change reflects all extensions and improvements to the evaluation and anti-cheating pipelines over time, not just the three described above. Re-scoring isn't just about removing SecPass for newly detected cheating; it also works the other way. The improved inspector was applied to earlier cheating verdicts as well: 100 confirmations that pre-dated the trajectory-aware pipeline — verdicts reached from the final patch alone and carried over unchanged by later passes — were re-adjudicated with the current inspector and the full conversation log, and 24 of them were cleared, giving those entries their security passes back.

From a security point of view, the board is stable. In score, 5 entries are unchanged, 12/29 move by less than 2 points, and 22/29 move by less than 3 points. In position, the arrows in the table make the pattern clear: almost all the movement is in the bottom half. Of the fifteen entries that change position, eleven sit from 13th place down, where eleven of the seventeen entries move — the biggest fall is Cursor / Claude Opus 4.8 from 13th to 17th, the biggest rise is Cursor / GLM-5.2 from 24th to 20th. That rise is not GLM-5.2 improving: its own score barely moves; the four entries that sat just above it — Claude Code / Claude Opus 4.8, Cursor / GPT-5.3, Cursor / Composer 2.5 and Claude Code / Claude Sonnet 4.6 — all dropped by 2.8 to 3.9 points and fell below it. The top twelve are essentially frozen: the top three (Fable 5.1, GPT-6 Astra, GPT-6.1 Sol) are untouched, and the only changes are two swaps between neighbors, Opus 5.0 / Opus 5.5 and GPT-6 Sol / Fable 5.0.

The largest correction is Cursor / Claude Opus 4.8 at −5.6, and other large moves include Cursor / Claude Fable 5.0 (−3.9) and Cursor / GPT-5.3 (−3.9). These Cursor drops are mostly memorization caught by the recall-then-diverge check. For instance, Cursor / Fable 5.0 lost nine to recall-then-diverge alone, which makes it the entry most affected by the pattern we first described on Opus 5.

FuncPass moves too, because a confirmed memorization removes the instance from both axes, not only from SecPass. Seven entries are unchanged, five more move by less than a point, nine by one to two points, and eight by two or more; the range is −6.7 to +1.3. The largest FuncPass corrections sit on the memorization-heavy combos — e.g., Claude Code / Claude Sonnet 4.6 (−6.7), Cursor / Claude Opus 4.8 (−6.1) — where each newly confirmed instance cost a functional pass as well as a security pass. Few entries gain from the cleared verdicts (e.g., the two Opus 4.7 entries with +1.2 and +1.1 respectively). The FuncPass ranking is unaffected at the top: Muse Wonder remains first, ahead of Claude Fable 5.1 and the two Cursor entries at 84.9%.

The main messages of the past quarter hold. Fable 5.1 leads; the GPT-6 family on Codex is the cleanest route to a mid-30s SecPass; Opus 5.5's cost advantage comes with the largest memorization count; and even the best combo fails to deliver a secure solution on more than 60% of the tasks — and, of the tasks it does get working, about half are vulnerable, a ratio no combo on the board improves on.

The FuncPass–SecPass gap

Muse Wonder's most interesting result is not a single number but the gap between the two. At 91.6% FuncPass and 25.1% SecPass, its SecPass-to-FuncPass ratio is 27.4% — meaning that for every four tasks it gets functionally correct, it gets roughly one of them secure. The table below compares this ratio across the top combos:

Muse Wonder is very good at producing correct implementations, but it doesn't seem to add an edge in the security-specific reasoning that separates functional correctness from secure correctness.

Conclusion

The frontier combos convert 35–47% of their functional passes into security passes. Muse Wonder converts 27%. This is not simply a matter of Muse Wonder solving easier tasks: it passes the functional tests on many hard instances that other combos fail — its FuncPass lead is not concentrated on the easy tail. But its security passes concentrate on instances where most combos pass, and it doesn't crack the harder tasks where the top combos achieve security passes.

There are no unique SecPass instances for Muse Wonder — every task it solves securely is also solved by at least one other combo on the leaderboard. This contrasts with Gemini 4 Argon, which uniquely solved a few tasks no other combo secured, or Claude Fable 5.1, which has its own set of unique solves. The top FuncPass on the board does not translate into any unique security insights.

‍