Muse Spark 1.2 Benchmarks vs Claude Opus 5: Do the Claims Hold Up?

When a vendor launches a model, the benchmark chart is a marketing asset. Reading it well means asking three questions: who ran it, who was in the comparison set, and does anyone independent get the same number?

For Muse Spark 1.2, launched alongside Muse Code on August 5, 2026, the answers are unusually interesting — because Meta published a chart on which it loses.

What did Meta actually claim?

Three coding benchmarks, all run by Meta, all showing Muse Spark 1.2 in second place behind Claude Opus 5.

A fourth chart, covering kernel-generation work, places Muse Spark 1.2 fourth of six — so across every benchmark Meta chose to publish, it leads none of them. Meta's methodology note also concedes the comparison is "not harness-identical to the leaderboard" and "may not reflect these models' best performance."

Publishing a chart you lose is a defensible choice — it signals confidence that the price-to-performance ratio is the real argument. But it does mean the headline is settled: Muse Spark 1.2 is not the strongest coding model available. Meta says so.

The generational improvement is real, though. Against Muse Spark 1.1, that is +6.7 points on Terminal-Bench and +6.3 on DeepSWE in roughly four weeks. That pace is the thing competitors should be watching, not the absolute position.

Who was left out of the comparison?

This is where the chart gets less generous.

Meta benchmarked against GPT-5.6 Terra — OpenAI's mid-tier model — rather than Sol, the top of the range. Sol scores meaningfully higher on Terminal-Bench than any model in Meta's table. Swapping in the flagship would have pushed Muse Spark 1.2 to third.

Claude Fable 5 is also absent from the coding charts, and the Meta Internal Coding Bench row omits several competitors entirely.

None of this is unusual — every vendor picks its comparison set — but it is the difference between "second-best coding model" and "second-best among the models Meta chose to include."

Does independent testing agree?

No, and the gap is large enough to matter.

The core problem with vendor benchmarks is the harness. A benchmark score measures a model plus the agent scaffold wrapped around it. Vendors run their own model inside their own optimised harness, which inflates results relative to a neutral setup.

Vals AI runs every model through one common harness. On that basis Muse Spark 1.2 ranks 14th out of 50 on Terminal-Bench — against 2nd place on Meta's vendor-agent configuration. That is not a rounding difference; it is a different conclusion.

Artificial Analysis measures Terminal-Bench at 80%, not 82.9%, and puts the 1.1 to 1.2 improvement at roughly +2 points rather than Meta's +6.7.

Muse Spark 1.2 does not currently appear on the official verified Terminal-Bench leaderboard at all.

Is there a track record to judge this against?

Yes, and it is the most useful single data point in this whole exercise.

At the Muse Spark 1.1 launch in July 2026, Meta claimed 80.0% on Terminal-Bench 2.1. Independent verification subsequently returned 76.2% ± 1.2.

The claimed figure sat above the upper bound of the verified confidence interval. Not catastrophically — but it is a systematic direction, not noise. And note that Meta's own launch table for 1.2 lists Muse Spark 1.1 at 76.2%, the verified number, not the 80.0% it originally claimed. The generational gain is being measured against a corrected baseline while the new number remains unverified.

Applying the same correction informally to 1.2 would land it near 79%, which is roughly where Artificial Analysis independently puts it. Two independent methods converging on the same discount is worth noting.

To Meta's credit, its published methodology concedes the harness "may not reflect these models' best performance" — a caveat most vendors omit.

How verifiable are Meta's numbers?

Less than you would want. Meta published its benchmark results as chart images rather than as tabular data, and the accompanying methodology document is largely a table of contents. There is no released harness configuration, no per-task breakdown and no raw results file — so the figures cannot be independently recomputed, only re-run from scratch by a third party using a different scaffold.

The clearest illustration of why that matters: Muse Spark 1.1 now has three different Terminal-Bench 2.1 scores in circulation — 76.2%, 78% and 80.0% — depending entirely on who ran it and in which harness. Same model, same benchmark, a four-point spread. When a single model produces that much variance, a 3.8-point gap between two different models is not a reliable ranking signal.

Where does Muse Spark 1.2 genuinely win?

Cost efficiency, and it is not close.

On evaluations that measure cost alongside capability, Muse Spark 1.2 is the cheapest model in the top tier — roughly $0.69–0.70 per test against competitors several times that. It ranks around 5th of 45 on blended intelligence-per-dollar indices.

It also posted the launch cycle's largest jump on GDPval-AA v2, a general knowledge-work evaluation, climbing about 260 Elo points to roughly 1631 and 5th place.

Throughput is strong too: around 165 tokens per second on standardised measurement, with real-world tests reporting averages near 191.

What did not improve: raw knowledge went slightly backwards, with small declines on SciCode and Humanity's Last Exam. This is a coding-specialised point release, and the specialisation shows in both directions.

What does this look like in practice?

The most credible independent hands-on test so far captures both sides precisely.

On the upside: Muse Code audited 222 pull requests in under five minutes for about ten cents, against roughly $32 for the same job on a frontier model. That is a genuine order-of-magnitude shift in what is economically sensible to automate.

On the downside: in the same session, the agent spent three minutes researching a Google project that does not exist, then built its entire integration plan on that fabrication and could not recover. The tester's verdict — that you cannot trust it for longer-running work — is the single most useful sentence written about the release.

Those two results are consistent with each other, and with the benchmarks. Bounded, verifiable, high-volume tasks are where a cheap, fast, slightly-weaker model wins outright. Long-horizon autonomous work, where a single early error compounds across hours, is where the capability gap gets expensive.

How should you read the numbers?

• Treat vendor benchmarks as an upper bound. Vendor-run scores measure the model plus a harness tuned for it. Expect a few points of regression on a neutral setup.
• Check the comparison set before the bars. Terra rather than Sol changes the ranking without changing a single number.
• Weight verified over claimed. Where independent harnesses exist, they are the number to plan around.
• Benchmark deltas under 5 points rarely survive contact with your codebase. Your language, repo size, test coverage and prompts move results more than the gap between second and fourth place.
• Measure cost per solved task, not per token. A weaker model burns more turns. Once you account for retries, a 12–21x token discount compresses to low single digits on real work.

For the wider leaderboard picture, see our AI agent benchmark roundup.

The verdict

Muse Spark 1.2 is a strong second-tier coding model priced like a budget one. It is not the frontier, by Meta's own accounting, and independent harnesses put it further from the frontier than Meta's chart suggests.

That is a perfectly good product. The rate of improvement — a meaningful jump in about four weeks — is more strategically significant than the current position. But if you are choosing a model on capability alone for long-horizon agentic work today, the benchmarks point at Claude Opus 5, and they were published by Meta.

For what that means in daily use, see Muse Code vs Claude Code.

FAQ

Is Muse Spark 1.2 better than Claude Opus 5?

No. On all three coding benchmarks Meta published at launch, Claude Opus 5 scored higher — 86.7% vs 82.9% on Terminal-Bench 2.1, 65.0% vs 59.3% on DeepSWE, and 79.4% vs 70.6% on Meta's internal benchmark.

What is Muse Spark 1.2's Terminal-Bench score?

Meta claims 82.9% on Terminal-Bench 2.1. Artificial Analysis independently measures 80%, and Vals AI's common harness ranks it 14th of 50. It is not yet on the official verified leaderboard.

Why do independent benchmarks differ from Meta's?

Benchmark scores measure a model plus its agent harness. Vendors optimise their own harness, which inflates results relative to a neutral common scaffold like the one Vals AI uses.

Did Meta overstate Muse Spark 1.1's benchmarks?

Meta claimed 80.0% on Terminal-Bench 2.1 for Muse Spark 1.1; independent verification returned 76.2% ± 1.2, above the confidence interval's upper bound. Meta's 1.2 launch chart now lists 1.1 at the verified 76.2%.

What is Muse Spark 1.2's context window?

1,048,576 tokens (1M), with maximum output around 131,072 tokens. Meta has not published the architecture, parameter count or knowledge cutoff.

Is Muse Spark 1.2 good value?

Yes, on cost efficiency it is the standout — roughly $0.69–0.70 per test, the cheapest in the top tier. Its case is price-to-performance, not raw capability.

Can I download Muse Spark 1.2 weights?

No. It is closed-weights, with no self-hosting or fine-tuning, continuing Meta's shift away from the open-weight Llama approach.

Should I switch to Muse Spark 1.2 for coding?

For bounded, high-volume, verifiable work where token cost dominates — yes, the economics are compelling. For long-horizon autonomous work, the capability gap and documented hallucination failures argue for a frontier model.