How to Run Muse Glimmer Locally: Ollama, LM Studio, llama.cpp and vLLM

The fastest way to run Muse Glimmer locally is ollama run muse-glimmer (or muse-glimmer:30b-mlx on Apple Silicon), on a machine with 24–32GB of GPU or unified memory. The 4-bit build is about 17GB, and Meta says it fits under 20GB. LM Studio, llama.cpp, MLX, vLLM and SGLang all had support at launch, and every one of them exposes an OpenAI-compatible endpoint you can point an agent at.

Muse Glimmer 30B is Meta's open-weights model, released August 10, 2026 under Apache 2.0 and distilled from Muse Spark (model card). It's a dense ~29.6B model with a ~1.8B vision encoder, a 131,072-token context and text plus image input.

What hardware do you need for Muse Glimmer?

Build	Size	Fits on	Source
4-bit (Q4 / NVFP4)	~17GB	24GB GPU, 32GB+ Mac	Unsloth
BF16	~58–60GB	80GB GPU, 64GB+ Mac	Unsloth, model card

Meta targets M4/M5 Max Macs, the RTX 5090 and other 24–32GB devices (Meta).

Context eats memory too. Sebastian Raschka calculates about 52 KiB of KV cache per token, thanks to only 2 KV heads. At the full 131K context that's roughly 6.5GB on top of the weights. Alok fit a 130K context in 19.3GB on one RTX 4090 without KV-cache quantization.

Smaller cards work with compromises:

• 12GB: an EXL3 3.00bpw quant ran fully in 12GB VRAM at 100K context, about 30 tok/s, as reported by the author.
• 8GB: Q4_K_M via llama.cpp on an RTX 4060 laptop, with partial offload to system RAM.
• Edge: 2.84 tok/s on a Qualcomm Dragonwing board at 21.6GB resident.

Option 1: Ollama

Ollama shipped Glimmer on day one (Ollama):

ollama run muse-glimmer
# Apple Silicon, MLX engine
ollama run muse-glimmer:30b-mlx

Ollama says the MLX tag runs 1.5–1.8x faster with DFlash. It also wires Glimmer straight into coding and assistant harnesses:

ollama launch claude --model muse-glimmer
ollama launch pi --model muse-glimmer
ollama launch hermes --model muse-glimmer

davidlee demoed exactly this, Glimmer behind Codex and Claude Code. The Ollama library entry lists the tags.

Option 2: LM Studio

In LM Studio, open the model catalog, search for Muse Glimmer and download it. To serve it to other apps, start the local server with lms server start (default port 1234) or enable it in settings (LM Studio). One user ran the official 17GB GGUF at 64K context under 20GB of VRAM to critique his photos.

LM Studio reports Glimmer completed 83.3% of its 18-task BionicBench v0.1, against 77.7% for Gemma 4 31B and Qwen 3.6 27B.

Option 3: llama.cpp

GGUF quants are listed on Hugging Face under the model's quantized variants, and llama.cpp had day-0 support. Serve with llama-server and point clients at its OpenAI-compatible endpoint. Worked examples:

• CogniTech's repo runs Glimmer on an M4 Max with llama.cpp and wires it into Claude Code via LiteLLM, benchmarking with and without speculative decoding.
• Mozilla.ai packaged it as a single llamafile driving the Hermes agent, which opened a real draft PR.
• filipe's GB10 recipe pins weights and a llama.cpp build for 29.6 tok/s on a DGX Spark.

Option 4: vLLM or SGLang for serving

For a shared server, the model card gives:

pip install vllm
vllm serve "meta-models/Muse-Glimmer-30B"

pip install sglang
python3 -m sglang.launch_server --model-path "meta-models/Muse-Glimmer-30B" --host 0.0.0.0 --port 30000

SGLang reports about 230 tok/s on one RTX 5090 with NVFP4 and DFlash. For AMD, one builder serves it on a single MI300X with a custom ROCm vLLM image.

MLX on Apple Silicon

Many community MLX conversions on Hugging Face wouldn't load at first. muse-glimmer-mlx supplies the missing runtime, and the MLX playground is a local vision-and-reasoning chat for high-memory Macs. Ollama's MLX tag is the easiest route.

Settings that matter

The model card recommends temperature 1.0, top-p 0.95, top-k 64. Reasoning strength goes in the system prompt as Reasoning strength: high, with low, medium, high or xhigh. Meta suggests high or xhigh for coding and agent tasks.

For images, Glimmer has a 4,096 visual-token limit. Glimmer HD Vision is a proxy that splits a 4K image into an overview plus four detail tiles.

Does DFlash speculative decoding help?

Sometimes. Meta reports 3.1x on an RTX 5090 (74.9 to 233.4 tok/s) and 1.8x on an M5 Max (26.6 to 50.2 tok/s). But one Reddit user measured only a 0.9–1.2% speedup on an M5 Pro through oMLX. Test it on your hardware before assuming the headline number.

What to expect from a local Glimmer agent

Glimmer's reputation is strong tool calling and recovery, weaker open-ended generation. AICodeKing found it strong on multi-step tasks, and Simon Willison found multi-step reasoning and tool use strong but creative output mixed. A controlled M1 Max test had it pass 24/30 versus Qwen 3.8's 30/30, and a Dell GB10 report was a documented negative result. Run a small pilot on your own tasks.

To fine-tune, Unsloth offers free notebooks that run on 24GB. More local builds are in research and data.

Frequently asked questions

How much VRAM does Muse Glimmer 30B need?

The 4-bit build is about 17GB and Meta targets 24–32GB devices. BF16 needs about 58–60GB. Community quants have run it in 12GB and even 8GB with offloading, at lower speed or quality.

Can Muse Glimmer run on a Mac?

Yes. Use ollama run muse-glimmer:30b-mlx or LM Studio on an Apple Silicon Mac with enough unified memory. Meta names the M4 and M5 Max as targets, and 32GB or more is the practical floor for 4-bit with useful context.

How do I use Muse Glimmer with Claude Code?

Run ollama launch claude --model muse-glimmer, or serve it with llama.cpp and route Claude Code through LiteLLM as shown in the CogniTech repo. Expect lower quality than frontier models on hard tasks.

Is Muse Glimmer free for commercial use?

It's released under Apache 2.0, which permits commercial use. Check the license file on the Hugging Face repo for the exact terms.

How do I set the reasoning level in Muse Glimmer?

Add a line such as Reasoning strength: high to the system prompt. Supported values are low, medium, high and xhigh.