Running Muse Glimmer on a Consumer GPU: What Builders Report

Muse Glimmer 30B runs comfortably on a 24GB GPU at 4-bit, and builders have pushed it down to 12GB and even 8GB cards with smaller quants and offloading. Unsloth's 4-bit build is about 17GB, one builder fit a 130K-token context in 19.3GB on an RTX 4090, and another ran 100K context at about 30 tok/s on a 12GB laptop GPU. DFlash speculative decoding can triple speed on an RTX 5090 by Meta's numbers, but results on Macs vary a lot.

All speeds below are as reported by the authors, on their own setups. Compare them loosely.

How much memory does Muse Glimmer need?

Meta says Glimmer fits in under 20GB at 4-bit and targets 24 to 32GB devices such as M4/M5 Max Macs and the RTX 5090 (Meta). Unsloth's size table (Unsloth):

Quant	Recommended memory
2-bit (UD-Q2_K_XL)	12–14 GB
3-bit (UD-Q3_K_XL)	14–15 GB+
4-bit (UD-Q4_K_XL, NVFP4)	17 GB+
6-bit (UD-Q6_K_XL)	20–22 GB+
8-bit (UD-Q8_K_XL)	34 GB+
BF16	58 GB+

Context is cheaper than you might expect. Sebastian Raschka's architecture notes put the KV cache at about 52 KiB per token, thanks to 2 KV heads and 3:1 sliding-window to global attention. Default context is 131,072 tokens; Unsloth lists up to 262,144.

NVIDIA GPU reports, from 5090 to 8GB laptops

Hardware	Setup	Reported result	Source
RTX 5090 (32GB)	Meta's test, DFlash	74.9 to 233.4 tok/s (3.1x)	model card
RTX 5090	SGLang, NVFP4 + DFlash	about 230 tok/s	SGLang
RTX 4090 (24GB)	No KV-cache quantization	130K context in 19.3GB VRAM, 3,100+ tok/s prefill	Alok
RTX 3090 (24GB)	Two agent harnesses compared	Speed and VRAM measured on video	NetworkCoder
GPU not stated, LM Studio	Official 17GB GGUF	64K context under 20GB VRAM	松xR
16GB single GPU	Performance, memory, agent tests	Full build tests on video	Luke's Dev Lab
12GB laptop GPU	EXL3 3.00bpw, fully in VRAM	100K context, about 30 tok/s	PyaesoneP
RTX 4060 laptop (8GB)	Q4_K_M via llama.cpp	Ran bug-fixing, tool recovery and invoice extraction tests	Tarun Singh

The 12GB report is the most interesting for mid-range owners. The author credits EXL3 quantization, and the post's title is a plug for it. The 8GB report does not claim a speed. A Q4_K_M build of a 30B model is far larger than 8GB, so much of it has to sit in system RAM; plan for it to be slower than the cards above.

Mac reports and the DFlash question

Hardware	Setup	Reported result	Source
M5 Max	Meta's test, DFlash	26.6 to 50.2 tok/s (1.8x)	model card
M5 Pro MacBook	4-bit via oMLX, with and without DFlash	0.90% to 1.16% speedup	OkSea7809
M5 Max, 128GB	Local chat and instruction tests	"Impressive" instruction following	Ashutosh Shrivastava
M4 Max MacBook	llama.cpp, with and without speculative decoding, wired into Claude Code	Benchmarked in repo	CogniTech Systems
M1 Max	Tool-calling and data-science benchmark	24/30 tasks passed	Ricky Zhou

The gap between Meta's 1.8x and one builder's roughly 1% on an M5 Pro is large. It could come down to the runtime (oMLX versus Meta's setup), the chip, or the workload. Ollama says its muse-glimmer:30b-mlx tag runs 1.5 to 1.8x faster with DFlash on Apple Silicon (Ollama). If speed matters, measure on your own prompts with DFlash on and off.

MLX support was patchy at first. muse-glimmer-mlx exists because many MLX conversions on Hugging Face would not load without a missing runtime. The MLX playground and muser, a standalone Metal engine, are other options.

DGX Spark and other small boxes

The NVIDIA DGX Spark and GB10-based machines are not consumer GPUs, but they are desk-sized and several builders use them:

• filipe's GB10 recipe went from 11.9 to 29.6 tok/s with DFlash (2.49x), with pinned weights and a llama.cpp build.
• Rajendra Rawat's NVFP4 tuning notes report 27.5 tok/s on average and 90/100 on his tool eval at 128K context.
• Ryan's reboot-durable stack runs Glimmer in Open WebUI.
• Ryan Gu's Dell GB10 report is a useful negative result: top vision and SRE-ops scores, but five failed deployment gates against DeepSeek V4 Flash.

At the far end, Samuel Alexander ran it on a Qualcomm Dragonwing IQ-9075 board at 21.6GB resident with full 131K context and 2.84 tok/s. Slow, but entirely on-device.

Which runtime should you use?

Glimmer had day-0 support in Ollama, LM Studio, llama.cpp, MLX, vLLM, SGLang and ExecuTorch (Meta). A rough guide from the reports above:

• Easiest: ollama run muse-glimmer, or LM Studio with the official GGUF.
• Most control on NVIDIA: llama.cpp with a chosen GGUF quant, or EXL3 on smaller cards.
• Serving several users: vLLM or SGLang, with NVFP4 on Blackwell.
• Mac: Ollama's MLX tag or an MLX port, and test DFlash yourself.

Once running, most people point an agent harness at it. Mozilla.ai ran it as a single llamafile and it opened a real pull request. Others use it behind Codex and Claude Code. See the research and data shelf for more benchmarks, and fine-tuning Glimmer if you want to adapt it.

Frequently asked questions

Can Muse Glimmer run on an RTX 4090?

Yes. One builder fit a 130K-token context in 19.3GB of VRAM on a single RTX 4090 without KV-cache quantization and measured over 3,100 tok/s prefill. A 4-bit build is about 17GB on its own.

Can I run Muse Glimmer on 12GB of VRAM?

One builder runs it fully in 12GB using an EXL3 3.00bpw quant at 100K context and reports about 30 tok/s. Unsloth's 2-bit GGUF is listed at 12 to 14GB.

Does DFlash speed up Muse Glimmer on a Mac?

Meta reports 1.8x on an M5 Max and Ollama claims 1.5 to 1.8x with its MLX tag, but one M5 Pro tester measured only about 1% using oMLX. Results depend on runtime and chip, so benchmark it yourself.

What is the fastest reported Muse Glimmer speed on a single consumer GPU?

On an RTX 5090, Meta reports 233.4 tok/s with DFlash and SGLang reports about 230 tok/s with NVFP4 and DFlash. Both are vendor-reported figures.