Meta Muse Spark 1.1 Tops SWE-bench Pro: What the Leaderboard Actually Means (September 2026)
As of September 14, 2026, Meta's Muse Spark 1.1 sits at the top of Scale's SWE-bench Pro standardized public set at 61.5 percent and its private commercial set at 51.5 percent, the first model to lead both boards at once. It took the top spot from GPT-5.4 (xHigh) at 59.1 percent and Claude Opus 4.6 at 47.1 percent. That sounds decisive until you notice that three separate numbers, 61.5 percent, 80.0 percent, and 51.5 percent, are all being quoted somewhere as the best SWE-bench Pro score right now, and most pages that cite one never say which board they mean. This post pulls those numbers apart, explains what SWE-bench Pro actually rewards, and works out whether a leaderboard swap should change what your engineering team runs in production.
Meta Muse Spark 1.1 Tops SWE-bench Pro: What the Leaderboard Actually Means (September 2026)
As of September 14, 2026, Meta's Muse Spark 1.1 leads Scale's SWE-bench Pro standardized public set at 61.5 percent and its private commercial set at 51.5 percent. That makes it the first model to sit at the top of both boards at the same time, a distinction that had not happened before. It displaced GPT-5.4 (xHigh), which held the public board at 59.1 percent, and Claude Opus 4.6, which topped the commercial set at 47.1 percent.
| Board | Muse Spark 1.1 | Previous leader | Gap |
|---|---|---|---|
| SWE-bench Pro, standardized public set | 61.5% | GPT-5.4 (xHigh), 59.1% | +2.4 pts |
| SWE-bench Pro, private commercial set | 51.5% | Claude Opus 4.6, 47.1% | +4.4 pts |
Two things are worth doing with this result. The first is understanding what the number rewards, because SWE-bench Pro is stricter and messier than the benchmark most people still picture when they hear "SWE-bench." The second is resisting the reflex to switch production models because a board reshuffled by a couple of points.
The Model Itself
Muse Spark 1.1 is Meta's agentic multimodal model, announced on July 9, 2026 alongside the public preview of the Meta Model API for US developers. It carries a 1 million token context window and up to 131K output tokens, accepts text, images, video, audio, and PDF documents, and is built around tool use, computer use, and coding. API pricing is $1.25 per million input tokens and $4.25 per million output tokens, with cache reads at $0.15 per million. That places it well below frontier closed models on sticker price, which matters for the cost-per-issue math later.
The relevant point for a coding leaderboard is that Muse Spark 1.1 was trained for agentic loops rather than single-shot completion. SWE-bench Pro rewards exactly that behavior, so a strong showing here is consistent with the model's design rather than a surprise.
What SWE-bench Pro Actually Measures
SWE-bench Pro is the harder successor to SWE-bench Verified, and the differences are the whole story. It uses a deterministic harness, so a run is not polluted by flaky tests that pass or fail at random. It draws from a larger and more diverse issue set. And it scores strictly: a patch either makes the hidden tests pass or it does not, with no partial credit for a diff that looks plausible. We went through the mechanics of why Pro and Verified disagree in our breakdown of the two benchmarks, and the summary is that Pro produces lower absolute pass rates but discriminates between models much better.
A leader on Pro is therefore telling you something more specific than a leader on Verified. It means the model resolves real repository issues, end to end, against tests it cannot see, inside an agent loop that has to read code, plan an edit, apply it, and verify. That is close to what a coding agent does in production, which is why Pro is the board worth tracking.
The Three Numbers Problem
Here is where the leaderboard becomes a trap. Three different figures are currently circulating as the best SWE-bench Pro score, and they are not measuring the same thing.
- 61.5 percent: Muse Spark 1.1 on Scale's standardized public set. This is the board where every model runs through the same scaffolding on the same public issue split.
- 51.5 percent: Muse Spark 1.1 again, on Scale's private commercial set. The issues are held back to prevent training contamination, so scores run lower and the ranking can differ from the public board.
- 80.0 percent: Claude Fable 5 in the llm-stats vendor aggregate. This is not the Scale standardized board at all; it collects vendor-reported numbers under vendor-chosen scaffolding, so it is not comparable to the two Scale sets.
The spread between 51.5 and 80.0 is almost entirely scaffolding and data splits, not raw model capability. When a page says a model "leads SWE-bench Pro," the useful follow-up is always the same: which board, which split, and whose scaffolding. A vendor-reported aggregate is a marketing surface until an independent harness reproduces it on a fixed split. The Scale standardized public set is the closest thing to an apples-to-apples comparison, which is why the Muse Spark 1.1 result at 61.5 percent is the one worth anchoring on.
Should This Change What You Run?
A 2.4 point lead on the public board is real, and against the commercial set the 4.4 point margin is more convincing because that split is harder to game. But a leaderboard position is an average across a fixed issue set, and your codebase is not that issue set. Two questions decide whether the swap matters for you.
The first is cost per resolved issue, not pass rate. At $1.25 and $4.25 per million tokens, Muse Spark 1.1 is cheap enough that even a modest pass rate can beat a pricier model on total cost to clear a backlog. Multiply the pass rate by your average token spend per attempt, including retries, and compare the cost to actually close an issue. That number reorders leaderboards constantly, which is the same lesson we drew comparing Fable 5 and GPT-5.5 on cost per resolved issue.
The second is behavior inside your harness. A model that tops SWE-bench Pro under Scale's scaffolding may behave differently under yours, because tool call formatting, retry policy, and context construction all move the result. The Muse Spark family has generally traded on cheap tokens and long context, which we looked at with Muse Spark 1.3, and 1.1 keeps that profile. If your agent rebuilds context on every turn, you lose the cache economics that make the low price compelling.
When This Applies to Your Stack
Treat a leaderboard swap as a prompt to re-run your own evaluation, not as a reason to switch by default. The durable pattern is to keep your agent harness model-agnostic, hold a fixed set of 50 recently closed issues from your own repository as a private test set, and run a quarterly bake-off that measures pass rate, tool call reliability, and cost per resolved issue on that set. When a new leader appears, you feed it into the same harness and read the number that predicts your production behavior rather than someone else's.
Contra Collective builds and maintains the agent infrastructure that makes those bake-offs tractable, from model-agnostic harnesses to the CI integration that lets a coding agent open and validate real pull requests. If your team wants a private SWE-bench-style evaluation wired to your own codebase so a leaderboard change becomes a one-hour test instead of a debate, that is work we can help structure.
FAQ
Is Muse Spark 1.1 actually the best coding model now?
It leads both SWE-bench Pro boards run by Scale as of September 14, 2026, which is the strongest standardized coding signal available. Whether it is best for you depends on cost per resolved issue and how it behaves in your harness, not on the leaderboard position alone.
Why do people quote 80 percent for SWE-bench Pro?
That figure comes from the llm-stats vendor aggregate, which collects vendor-reported scores under vendor-chosen scaffolding. It is not the Scale standardized board and is not comparable to the 61.5 percent public set number. Always ask which board a score refers to.
What is the difference between the public and commercial SWE-bench Pro sets?
The standardized public set uses issues that are openly available, so every model runs on the same known split. The private commercial set holds issues back to reduce training contamination, which lowers scores and can change the ranking. Muse Spark 1.1 currently leads both.
How much does Muse Spark 1.1 cost to run?
API pricing is $1.25 per million input tokens and $4.25 per million output tokens, with cache reads at $0.15 per million. For agentic coding, the number that matters is cost per resolved issue, which folds in retries and cache hit rate.
Should I switch my production agent to Muse Spark 1.1 today?
Not on the leaderboard alone. Run it through your own harness against a private set of your recently closed issues, measure cost per resolved issue and tool call reliability, and switch only if those numbers justify it.
Sources
More from the lab.
Claude Fable 5.2 vs GPT 6 Astra: Cheaper per Token, Pricier per Task (November 2026)
Claude Fable 5.2 launched November 9, 2026 as Anthropic's most capable generally available model, aimed squarely at GPT 6 Astra. Fable is cheaper on every list price line: 8 against 10 dollars input, 40 against 50 output, and 0.20 against 1.00 for cache reads. It also leads Astra on the Artificial Analysis Intelligence Index, 65 to 57, and on most published benchmarks. Then the per task numbers flip the story: at max effort Fable cost about four times as much per index task, because it writes far more tokens. Here is how to choose.
Claude Fable 5.2 vs Opus 5.5: Is Double the Price Worth It? (November 2026)
Anthropic shipped Claude Fable 5.2 on November 9, 2026, at exactly twice the price of Claude Opus 5.5 with the same context window and output limit. Fable leads 11 of 12 rows in Anthropic's launch table, but the size of the lead varies enormously: 3.6 points on FrontierCode 2.0, 16 on Terminal Bench Science, 20 on Remote Labor Index, 25 on RoboDojo. At matched high effort the two tie on agentic coding. Anthropic itself still recommends Opus 5.5 for everyday work. Here is how to decide which requests deserve the more expensive model.
Gemini 4 Carbon vs Fable 5.2, Opus 5.5, GPT 6 Astra, and Grok 4.7 (November 2026)
Google announced Gemini 4 Carbon on October 28, 2026 as a cyber defender model, then opened it to paid API developers and AI Ultra subscribers on November 10, a day after Anthropic launched Fable 5.2. Through November 30 it lists at 2 dollars input and 10 output, a quarter of Fable's price, rising to 4 and 20 after. It scores 60 on the Artificial Analysis Intelligence Index, behind Fable at 65 and ahead of Opus 5.5 at 59, Astra at 57, and Grok 4.7 at 46. Fable beats it on hard engineering; Carbon wins business automation at under half the cost. Here is the head to head against each rival.