Benchmarks & research

Alex Volkov

@altryne

Here's my full in depth review of @muse (muse.ai) Meta's new FREE AI agent with it's own browser, computer and connectors to anything from FB marketplace to Apple health! I go into setting it up, have it buy stuff securely with @link, and 3 great usecases that

thehype.

@thehypedotnews

muse glimmer 30b vs nemotron 3.5 lightning – on agentic coding two 30b open-weight agent models, released three days apart, both sold as the model you leave running. same harness, same tools, same tasks, two of them: repair a broken landing page, then write breakout from a spec

Better Stack runs both new 30B models on autonomous coding, Adobe Premiere control via MCP and parsing a 100+ page SEC filing.

wh

@nrehiew_

Muse Glimmer was trained directly logit distilled from Muse Spark. This means that there isn't a traditional 'base model' in that it was trained from the start on agentic traces. Super cool and haven't seen this approach in a while Welcome back GPT OSS

Arena.ai

@arena

Muse Spark 1.1 has entered the Code Arena: Frontend at #9! Muse Spark 1.1 reshapes the cost-performance Pareto Frontier by scoring 1541 at a blended $3.5M ($1.25 per input MToken, $4.25 per output MToken). This is frontier performance at a fraction of the price. Congrats to

Arena.ai

@arena

Muse Spark 1.3 Max by @AIatMeta has reshaped the Pareto frontier for Code Arena: WebDev! Meta's latest model at Max reasoning is doing something interesting on the Arena Pareto frontier: it's the only model holding down the wide price band between Qwen3.8-max ($5/MToken) and

Artificial Analysis

@ArtificialAnlys

Muse Glimmer's gaps against its class concentrate in agentic evaluations: 953 Elo on GDPval-AA v2 against 1141 for Qwen3.6 27B (Reasoning), 1141 for Gemini 3.5 Flash-Lite, and 1004 for Kimi K2.5 (Reasoning), with Terminal-Bench v2.1 (52%) also behind Qwen3.6 27B (61%). The

@JacobStephens2

@JacobStephens2

A reproducible evaluation of a stochastic defect where Muse Spark 1.1, under some agent-like request envelopes, wrote to claude-smoke.txt when asked for muse-smoke.txt; a neutral control kept 80/80 filenames intact.

I'm guessing that many people have been waiting for this comparison. For clarity, both models are running at full FP16 KV-cache. Due to VRAM limitations, Muse Glimmer is running full 262,144 context, whilst Qwen3.6 27B can only run at 147,500 context - full GPU offload in both cases. Both models have been coding on an enterprise-grade web application. Detailed report of each model (warning - includes AI generated content): Diagnostic quality - comparable. Both have shown genuinely good root-cause work when they apply themselves. Qwen found coding issue and worked to fix things cleanly. Muse Glimmer correctly traced bugs and even caught something that a Frontier model missed after more than 10 rounds of review. Neither one is weak at diagnosis. Implementation reliability - Qwen ahead. Qwen did introduce real regressions into the coding along the way (eg. severe zone-scope refactor regression, and case-sensitivity regression) but each one eventually got fixed properly once caught, usually within one or two corrective rounds. Muse Glimmer did land fixes that were clean and verified true to spec. However, when working in a complex environment exceeding 200k context, Muse Glimmer fa

AI at Meta

@AIatMeta

Today we’re also previewing WildArtifactBench, an internal evaluation framework designed to assess agents on complex, real-world tasks across diverse deliverable formats. By using win rates and Elo scores from human and agentic preference judges rather than strict ground-truth

Artificial Analysis

@ArtificialAnlys

Last week the Intelligence Index vs Cost Pareto frontier moved out substantially. Claude Fable 5.1, Muse Spark 1.3, and GPT-6 Astra each set a new point in efficient intelligence Link to analysis: artificialanalysis.ai/#intelligence-…

@accretional

@accretional

Unofficial knowledge base and tools for Meta's Muse Spark, starting with a root-cause write-up of Codex tool-calling failures with Muse Spark 1.1.

Cline

@cline

Muse Spark 1.1 just launched and it's their most capable coding agent model yet. On Terminal-Bench 2.1 it scores 80.0%, in the same cluster as Opus 4.8 (82.7%) and GPT 5.5 (83.4%). Use it in Cline with the Meta API!

@barkleesanders

@barkleesanders

A static teardown of Meta's Muse macOS app (codename Endo) finding a full computer-use agent in the public build, switched off by five server-delivered feature flags.

A harder rematch between Muse Glimmer and Qwen 3.8 27B (bumped to high reasoning) on the TalkWithMe project.

WorldofAI benchmarks Muse Glimmer 30B on consumer hardware with a self-built test harness and compares it against Qwen 3.6 27B.

Iam_

@SPAC89

Muse Spark 1.3 Ultra Contributor vs Fable 5.1 xHigh Same prompt, both ran for roughly 2 hours The prompt had a self improvement rule: if the independent judges scored the result below 9.5/10, it had to keep improving and try again, What shocked me most was that Muse Spark just

Evergreen Capital

@evergreencap3

I tested $META's Muse Spark over the last few hours and came away net positive. 3 main takeaways: 1) Quality: It's a very good model. Not quite frontier but good. It showed comparable performance vs Opus 4.6 across web data search, PDF parsing, and general

ThursdAI hosts compare daily-driver assistants, then David Pawlan, who ran 273 tests across 23 assistants for Assistant Benchmark, walks through how Muse stacks up.

Derya Unutmaz, MD

@DeryaTR_

Here is a crazy Jev @typesafeai example that I’m betting nobody has thought about: I asked @Muse from @Meta to use Jev to select the top 100 unanswered questions in immunology from 10,000 literature-grounded candidates. A few minutes later, it came back with some of the best

Trapit Bansal

@TrapitBansal

We entered Meta models in five international STEM Olympiads, as an uncontaminated eval of their reasoning capabilities. Three of these were live participations and graded officially. The models achieved gold-medal results in all five! 1/5

Meta's engineering write-up on Muse security: isolated VMs, a separate Sentinel permission authority, credential surrogation and layered prompt-injection defenses, with bug bounties up to $300,000.

@tanishq-dubey

@tanishq-dubey

A reproducible Apple Silicon harness that runs six fixed quality tasks against MLX quantizations of Muse Glimmer 30B and records scores, tokens per second, peak memory and load time.

AI at Meta

@AIatMeta

To understand whether we're making genuine progress on reasoning, we entered our AI models in five STEM Olympiad competitions. The results: 🏅 Asian Physics Olympiad (APhO): Perfect score, theory exam 🏅 International Physics Olympiad (IPhO): Perfect score, theory exam 🥇 ht

Mark Zuckerberg

@finkd

Muse Voice Transcribe is MSL's first real-time audio perception model -- rolling out today. SOTA in streaming speech-to-text, it handles speaker diarization, and endpointing natively in a single model.

Povilas Korop scores Muse Spark 1.3 on six real coding projects (Laravel, React-TS, PHP, Flutter, Go): Max effort ranks #26 with 48.43/60 at about $0.01 per prompt and 3:42 per prompt.

J A Z I I

@notjazii

meta muse just mogged fable 5.1 tested fable 5.1 and muse spark 1.3 with same prompt at highest reasoning available and results came out really different > muse spark 1.3 completed task in one minute and costed almost nothing > fable 5.1 completed task in 70 minutes and costed

khaled

@eltokh7

Muse Spark 1.1 seems like a very good model. I tested it on Stata Benchmark and it ranks 4th (!) beating Opus 4.7/4.8, tied with GPT 5.5. I like this benchmark because it often catches seemingly good model performing poorly on OOD tasks

Taylor Arndt tested Muse with VoiceOver and found non-standard text fields and missing heading structure in chats, plus connector gaps such as iCloud email that kept it out of her work.

Arena.ai

@arena

Exciting news: Meta’s Muse Image just claimed #2 in the Image Arena! Muse Image from @AIatMeta now ranks second only to OpenAI's GPT Image 2, outperforming Nano Banana, Grok Imagine, MAI Image, and many other leading image models. It holds #2 across the board: Text-to-Image,

Box's Complex Work Eval finds Muse Spark 1.1 up to 5-6 points above the top-tier composite on structured work, and nearly 30 points ahead on cost-optimization analysis.

Ornith does really well. TielCoder (https://llm-bench.io/benchmarks/cmt7kp2zj002r01lcmpchvlko) might be even a bit better in coding. Will give it a try soon. Details of the comparison see here: https://llm-bench.io/compare/runs?runs=cmt6ecf8g000001p45vwzux53%2Ccmt6ergk5000701p41hqdyy78%2Ccmt6f2oob000e01p49o9592cb%2Ccmt6fqddm000l01p4l1vm7skd

Morgan

@morganlinton

Okay, finally finished my Muse Spark 1.3 benchmark. This ended up being the longest-running benchmark I've ever done at @VulcanBench and it definitely looks like Muse has some serious issues at lower effort levels. I was able to run Astra through my v4 eval suite across every

+1

elie

@eliebakouch

extremely exciting to see meta getting back into open weight models with a 30B dense first, and soon muse spark 1.2 they used knowledge distillation from muse spark, architecture wise it's similar to gemma 4 (which is llama 3 + swa (again!) + vision encoder), with scale free QK

Dan

@DanDr1s

Meta’s Muse Spark 1.3 just scored 62 on Artificial Analysis’ Intelligence Index. That ties Claude Fable 5, but Muse costs 8x less for input and nearly 12x less for output. It also scores above GPT-5.6 Sol, Grok 4.6, Kimi K3, and Gemini 3.8 Flash. Meta is suddenly in the top

pilvar (Philippe Dourassov)

@pilvar222

We ran Muse Spark 1.3 on our Cybersecurity benchmark, and it's actually not that good (yet) 😬 - At pass@1, it rediscovers an average of 19/32 CVEs. In comparison, Grok 4.6 scores 23.3/32 - When pooling the results of 3 runs (pass@3), Muse scores 24/32. DeepSeek V4 Pro gets

Arena.ai

@arena

Muse Spark 1.2 (xHigh) by @AIatMeta is now in Agent Arena, with a net improvement of +2.1%! Agent Arena measures models on millions of real-world, long-horizon agentic tasks. We use causal tracing methodology to measure a model's net improvement, indicating how much it improves

@ezcat207

@ezcat207

A curated, source-linked catalog of 156 real things people have done with the Muse personal agent, grouped into 16 categories with every entry linked to its original X post.

AICodeKing tests Muse Spark 1.3 and Gemini 3.8 Flash on frontend, 3D, SVG, maths and agentic coding, and flags file-overwriting issues.

RepoChad examines Muse Spark 1.3's 1,048,576-token context, DeepSWE and TerminalBench results, and the Max vs x-high reasoning modes.