Muse Spark 1.2 Benchmarks vs Claude Opus 5: Do the Claims Hold Up?

When a vendor launches a model, the benchmark chart is a marketing asset. Reading it well means asking three questions: who ran it, who was in the comparison set, and does anyone independent get the same number?

For Muse Spark 1.2, launched alongside Muse Code on August 5, 2026, the answers are unusually interesting — because Meta published a chart on which it loses.

What did Meta actually claim?

Three coding benchmarks, all run by Meta, all showing Muse Spark 1.2 in second place behind Claude Opus 5.

One row that chart cannot have: Grok 4.6, which xAI shipped on August 12, a week after Meta published. There is no Meta-run figure for it, and xAI's own launch numbers are not droppable into this column — xAI reports Terminal-Bench v3.0 (26%), a different and much harder version than the 2.1 scores above. Independent 2.1 runs of Grok 4.6 do exist (Artificial Analysis 88.39%, Vals 78.28%), but they come from different harnesses again, so they belong beside this table, not inside it.

A fourth chart, covering kernel-generation work, places Muse Spark 1.2 fourth of six — so across every benchmark Meta chose to publish, it leads none of them. Meta's methodology note also concedes the comparison is "not harness-identical to the leaderboard" and "may not reflect these models' best performance."

Publishing a chart you lose is a defensible choice — it signals confidence that the price-to-performance ratio is the real argument. But it does mean the headline is settled: Muse Spark 1.2 is not the strongest coding model available. Meta says so.

The generational improvement is real, though. Against Muse Spark 1.1, that is +6.7 points on Terminal-Bench and +6.3 on DeepSWE in roughly four weeks. That pace is the thing competitors should be watching, not the absolute position.

Who was left out of the comparison?

This is where the chart gets less generous.

Meta benchmarked against GPT-5.6 Terra — OpenAI's mid-tier model — rather than Sol, the top of the range. Sol scores meaningfully higher on Terminal-Bench than any model in Meta's table. Swapping in the flagship would have pushed Muse Spark 1.2 to third.

Claude Fable 5 is also absent from the coding charts, and the Meta Internal Coding Bench row omits several competitors entirely.

None of this is unusual — every vendor picks its comparison set — but it is the difference between "second-best coding model" and "second-best among the models Meta chose to include."

Does independent testing agree?

No, and the gap is large enough to matter.

The core problem with vendor benchmarks is the harness. A benchmark score measures a model plus the agent scaffold wrapped around it. Vendors run their own model inside their own optimised harness, which inflates results relative to a neutral setup.

Vals AI runs every model through one common harness. As of its August 12 update, Muse Spark 1.2 sits 5th on the Vals Index at 71.88% — behind Claude Fable 5, Claude Opus 5, Kimi K3 and GPT-5.6 Sol — while carrying the lowest cost per test in the top five, about $0.69. That is a strong showing for a budget model, and it climbed sharply in the week after launch.

The same board moved again on August 12, when Grok 4.6 entered 6th at 71.82% — 0.05 points behind Muse Spark 1.2, which is a tie in everything but sort order. Artificial Analysis reads the pair the other way round, putting Grok 4.6 at an Intelligence Index of 60.92 against Muse Spark 1.2's 57. When two neutral harnesses disagree about which of two models is ahead, the honest reading is that neither one is.

But the composite flatters it. On LiveBench it scores 78.0 overall with strong reasoning (90.0) and maths (91.2) — and just 57.6 on agentic coding, its weakest column against peers and a regression from Muse Spark 1.1's 58.5. Agentic coding is the one capability Muse Code is sold on.

Artificial Analysis currently places it at an Intelligence Index of 57, ranked 13th of 184 models, against Claude Opus 5 at 63.

Most importantly, Muse Spark 1.2 still does not appear on any verified agentic-coding leaderboard — not Terminal-Bench, SWE-bench, SWE-rebench or Aider. Its gains are all on composite and preference boards.

And the comparison cuts both ways, which is worth saying plainly: Claude Opus 5 is also absent from the verified Terminal-Bench board. The top verified Anthropic entry is Opus 4.8 at 78.9%. So Meta’s headline "82.9 versus 86.7" is vendor claim against vendor claim, not vendor against verified. Vals also discloses that both Opus 5 and Fable 5 used Claude Opus 4.8 as a refusal fallback in its Terminal-Bench runs; counting those nine affected passes as failures drops Opus 5 from 84.64% to 81.27%.

Is there a track record to judge this against?

Yes, and it is the most useful single data point in this whole exercise.

At the Muse Spark 1.1 launch in July 2026, Meta claimed 80.0% on Terminal-Bench 2.1. Independent verification subsequently returned 76.2% ± 1.2.

The claimed figure sat above the upper bound of the verified confidence interval. Not catastrophically — but it is a systematic direction, not noise. And note that Meta's own launch table for 1.2 lists Muse Spark 1.1 at 76.2%, the verified number, not the 80.0% it originally claimed. The generational gain is being measured against a corrected baseline while the new number remains unverified.

Applying the same correction informally to 1.2 would land it near 79%, which is roughly where Artificial Analysis independently puts it. Two independent methods converging on the same discount is worth noting.

To Meta's credit, its published methodology concedes the harness "may not reflect these models' best performance" — a caveat most vendors omit.

How verifiable are Meta's numbers?

Less than you would want. Meta published its benchmark results as chart images rather than as tabular data, and the accompanying methodology document is largely a table of contents. There is no released harness configuration, no per-task breakdown and no raw results file — so the figures cannot be independently recomputed, only re-run from scratch by a third party using a different scaffold.

The clearest illustration of why that matters: Muse Spark 1.1 now has three different Terminal-Bench 2.1 scores in circulation — 76.2%, 78% and 80.0% — depending entirely on who ran it and in which harness. Same model, same benchmark, a four-point spread. When a single model produces that much variance, a 3.8-point gap between two different models is not a reliable ranking signal.

Where does Muse Spark 1.2 genuinely win?

Cost efficiency, and it is not close.

On evaluations that measure cost alongside capability, Muse Spark 1.2 is the cheapest model in the top tier — roughly $0.69–0.70 per test against competitors several times that. It ranks around 5th of 45 on blended intelligence-per-dollar indices.

It also posted the launch cycle's largest jump on GDPval-AA v2, a general knowledge-work evaluation, climbing about 260 Elo points to roughly 1631 and 5th place.

Throughput is strong too: around 165 tokens per second on standardised measurement, with real-world tests reporting averages near 191.

What did not improve: raw knowledge went slightly backwards, with small declines on SciCode and Humanity's Last Exam. This is a coding-specialised point release, and the specialisation shows in both directions.

What does this look like in practice?

The most credible independent hands-on test so far captures both sides precisely.

On the upside: Muse Code audited 222 pull requests in under five minutes for about ten cents, against roughly $32 for the same job on a frontier model. That is a genuine order-of-magnitude shift in what is economically sensible to automate.

On the downside: in the same session, the agent spent three minutes researching a Google project that does not exist, then built its entire integration plan on that fabrication and could not recover. The tester's verdict — that you cannot trust it for longer-running work — is the single most useful sentence written about the release.

Those two results are consistent with each other, and with the benchmarks. Bounded, verifiable, high-volume tasks are where a cheap, fast, slightly-weaker model wins outright. Long-horizon autonomous work, where a single early error compounds across hours, is where the capability gap gets expensive.

How should you read the numbers?

• Treat vendor benchmarks as an upper bound. Vendor-run scores measure the model plus a harness tuned for it. Expect a few points of regression on a neutral setup.
• Check the comparison set before the bars. Terra rather than Sol changes the ranking without changing a single number.
• Weight verified over claimed. Where independent harnesses exist, they are the number to plan around.
• Benchmark deltas under 5 points rarely survive contact with your codebase. Your language, repo size, test coverage and prompts move results more than the gap between second and fourth place.
• Measure cost per solved task, not per token. A weaker model burns more turns. Once you account for retries, a 12–21x token discount compresses to low single digits on real work.

For the wider leaderboard picture, see our AI agent benchmark roundup.

The verdict

Muse Spark 1.2 is a strong second-tier coding model priced like a budget one. It is not the frontier, by Meta's own accounting, and independent harnesses put it further from the frontier than Meta's chart suggests.

That is a perfectly good product. The rate of improvement — a meaningful jump in about four weeks — is more strategically significant than the current position. But if you are choosing a model on capability alone for long-horizon agentic work today, the benchmarks point at Claude Opus 5, and they were published by Meta.

For what that means in daily use, see Muse Code vs Claude Code.

FAQ

Is Muse Spark 1.2 better than Claude Opus 5?

No. On all three coding benchmarks Meta published at launch, Claude Opus 5 scored higher — 86.7% vs 82.9% on Terminal-Bench 2.1, 65.0% vs 59.3% on DeepSWE, and 79.4% vs 70.6% on Meta's internal benchmark.

What is Muse Spark 1.2's Terminal-Bench score?

Meta claims 82.9% on Terminal-Bench 2.1, but that figure is vendor-run and Muse Spark 1.2 does not appear on the official verified Terminal-Bench leaderboard. On Vals AI's common harness it now sits 5th on the Vals Index at 71.88%, with the lowest cost per test in the top five. LiveBench scores its agentic coding at 57.6.

Why do independent benchmarks differ from Meta's?

Benchmark scores measure a model plus its agent harness. Vendors optimise their own harness, which inflates results relative to a neutral common scaffold like the one Vals AI uses.

Did Meta overstate Muse Spark 1.1's benchmarks?

Meta claimed 80.0% on Terminal-Bench 2.1 for Muse Spark 1.1; independent verification returned 76.2% ± 1.2, above the confidence interval's upper bound. Meta's 1.2 launch chart now lists 1.1 at the verified 76.2%.

What is Muse Spark 1.2's context window?

1,048,576 tokens (1M), with maximum output around 131,072 tokens. Meta has not published the architecture, parameter count or knowledge cutoff.

Is Muse Spark 1.2 good value?

Yes, on cost efficiency it is the standout — roughly $0.69–0.70 per test, the cheapest in the top tier. Its case is price-to-performance, not raw capability.

Can I download Muse Spark 1.2 weights?

No. It shipped closed-weights, but on August 10, 2026 Zuckerberg announced Meta will open-source Muse Spark 1.2's weights. They are not out yet and no date has been given. Meta did release Muse Glimmer, a 30B Apache-2.0 open-weights model, on the same day.

Should I switch to Muse Spark 1.2 for coding?

For bounded, high-volume, verifiable work where token cost dominates — yes, the economics are compelling. For long-horizon autonomous work, the capability gap and documented hallucination failures argue for a frontier model.