How to Run Muse Glimmer Locally: Ollama, LM Studio, llama.cpp and vLLM

To run Muse Glimmer locally, install Ollama and run ollama run muse-glimmer, or ollama run muse-glimmer:30b-mlx on an Apple silicon Mac. You need about 18GB free for the default 4-bit download, so a 24GB GPU or a 32GB Mac is the comfortable floor. LM Studio, llama.cpp, vLLM and SGLang work too, and all of them expose an OpenAI-compatible endpoint.

Muse Glimmer 30B is Meta's open-weights model, released August 10, 2026 under Apache 2.0 and distilled from Muse Spark (model card). It's a dense ~29.6B model with a ~1.8B vision encoder, a 131,072-token context, and text plus image input. Every command below is copied from the official source it's linked to.

Which runtime should you use?

Runtime	Best for	Command to start	Source
Ollama	Fastest setup, Mac or PC, plugging into coding agents	ollama run muse-glimmer	Ollama
LM Studio	A GUI and a local server without the terminal	Download in-app, then lms server start	LM Studio
llama.cpp	Choosing your own quant, DFlash, vision, max speed on one GPU	llama-server with a GGUF	Unsloth
vLLM / SGLang	Serving several users from one box	vllm serve "meta-models/Muse-Glimmer-30B"	Model card

Runtime choice matters more than it looks. In one M3 Max shootout, the same laptop ran Glimmer at about 29 tok/s on Ollama's MLX engine with DFlash, 21 on tuned llama.cpp and 10 on raw mlx-vlm. Across the 196 local-model entries in our catalog, llama.cpp or GGUF builds appear most (61), then vLLM and MLX (17 each). Real reports by GPU has the hardware breakdown.

How much VRAM does Muse Glimmer need?

Unsloth publishes sizes for each quant (Unsloth docs):

Quant	Memory	Example hardware (Unsloth)
2-bit (UD-Q2_K_XL)	12–14GB	RTX 4080
3-bit (UD-Q3_K_XL)	14–15GB	RTX 4090
4-bit (UD-Q4_K_XL, NVFP4)	17GB	32GB Mac
6-bit (UD-Q6_K_XL)	20–22GB	RTX 5090, 48GB Mac
8-bit (UD-Q8_K_XL)	34GB	128GB Mac, DGX Spark
BF16	58GB	128GB Mac, DGX Spark

Meta targets 24–32GB devices such as M4/M5 Max Macs and the RTX 5090, and says the 4-bit model fits under 20GB (Meta).

Context eats memory on top of the weights. Sebastian Raschka calculates about 52 KiB of KV cache per token, thanks to only 2 KV heads, which is roughly 6.5GB at the full 131K context. Real numbers from builders:

• RTX 5090, 64K context: DataCamp's tutorial used a 19.7GB dynamic quant and measured about 23.8GB of VRAM in use, leaving roughly 8GB free.
• RTX 4090, 130K context: Alok fit 130K context in 19.3GB without KV-cache quantization.
• 12GB laptop GPU: an EXL3 3.00bpw quant ran fully in 12GB at 100K context, about 30 tok/s, as reported by the author.
• 8GB RTX 4060 laptop: Q4_K_M via llama.cpp with partial offload to system RAM.
• 24GB M4 Pro MacBook: the 17GB GGUF runs with llama.cpp and Metal; the author documented the mistakes and fixes.

How to run Muse Glimmer with Ollama

Ollama shipped Glimmer on day one (Ollama blog). Install Ollama, then:

ollama run muse-glimmer
# Apple silicon, MLX engine
ollama run muse-glimmer:30b-mlx

The Ollama library lists three tags, all with 128K context and text plus image input:

Tag	Download size
muse-glimmer:latest	18GB
muse-glimmer:30b	18GB
muse-glimmer:30b-mlx	19GB

Ollama says the MLX tag runs 1.5–1.8x faster on Apple silicon with DFlash. That held up in one independent test: Paolo Rosson's M3 Max shootout got about 29 tok/s from Ollama's MLX engine, against about 21 from tuned llama.cpp and 10 from raw mlx-vlm.

Once it's running, Ollama serves an API on port 11434. The library page gives this test call:

curl http://localhost:11434/api/chat \
  -d '{"model": "muse-glimmer", "messages": [{"role": "user", "content": "Hello!"}]}'

Muse Glimmer with Claude Code, Codex and other agents

Ollama can launch coding agents with Glimmer as the model:

ollama launch claude --model muse-glimmer
ollama launch openclaw --model muse-glimmer
ollama launch hermes --model muse-glimmer
ollama launch pi --model muse-glimmer

davidlee demoed exactly this, Glimmer behind Codex and Claude Code. If you're deciding which coding agent to pair it with, see Muse Code vs Claude Code.

How to run Muse Glimmer in LM Studio

LM Studio added Glimmer in its Bionic app at launch. Per LM Studio's post: open Settings → Explore, find Muse Glimmer in the catalog and download it. To serve it to other apps, start the local server from the lms CLI with lms server start (LM Studio CLI docs). The server speaks OpenAI's Chat Completions format.

One user ran the official 17GB GGUF at 64K context under 20GB of VRAM to critique his photos. LM Studio reports Glimmer completed 83.3% of its 18-task BionicBench v0.1, against 77.7% for Gemma 4 31B and Qwen 3.6 27B.

How to run Muse Glimmer with llama.cpp

llama.cpp gives you the most control: pick your quant, add the vision projector, and turn on DFlash. These commands are from Unsloth's guide, which builds llama.cpp with CUDA (on a Mac, leave out -DGGML_CUDA=ON):

git clone https://github.com/ggml-org/llama.cpp
cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first \
  --target llama-cli llama-mtmd-cli llama-server llama-gguf-split
cp llama.cpp/build/bin/llama-* llama.cpp

Download the 4-bit GGUF and the vision projector:

hf download unsloth/Muse-Glimmer-30B-GGUF \
  --local-dir unsloth/Muse-Glimmer-30B-GGUF \
  --include "*mmproj-BF16*" --include "*UD-Q4_K_XL*"

Serve it with an OpenAI-compatible endpoint:

./llama.cpp/llama-server \
  --model unsloth/Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-UD-Q4_K_XL.gguf \
  --mmproj unsloth/Muse-Glimmer-30B-GGUF/mmproj-BF16.gguf \
  --temp 1.0 --top-p 0.95 --top-k 64 \
  --alias "unsloth/Muse-Glimmer-30B-GGUF" --port 8001

DataCamp's RTX 5090 run with this kind of setup measured 83.94 tok/s generation and about 121 tok/s on a coding task, then wired it into OpenCode. Their summary of Glimmer: fast at building, weaker at debugging.

Worked examples from the catalog:

• A starter script downloads Glimmer, serves it with llama.cpp on a Mac, connects OpenCode, and keeps the server on localhost only.
• CogniTech's repo runs Glimmer on an M4 Max and wires it into Claude Code via LiteLLM.
• Mozilla.ai packaged it as a single llamafile driving the Hermes agent, which opened a real draft PR.
• filipe's GB10 recipe pins weights and a llama.cpp build for 29.6 tok/s on a DGX Spark.

How to serve Muse Glimmer with vLLM or SGLang

For a shared server, the model card gives:

pip install vllm
vllm serve "meta-models/Muse-Glimmer-30B"

pip install sglang
python3 -m sglang.launch_server --model-path "meta-models/Muse-Glimmer-30B" --host 0.0.0.0 --port 30000

It also lists a Docker one-liner: docker model run hf.co/meta-models/Muse-Glimmer-30B. SGLang reports about 230 tok/s on one RTX 5090 with NVFP4 and DFlash. For AMD, one builder serves it on a single MI300X with a custom ROCm vLLM image.

Can Muse Glimmer run on a Mac?

Yes, if you have the memory. The easiest route is Ollama's MLX tag. Speeds vary a lot by chip:

• M5 Max: Meta reports 26.6 tok/s rising to 50.2 with DFlash, and Tim Darcet saw about 50 tok/s with the 17GB quant.
• M3 Max: about 29 tok/s through Ollama MLX, per the shootout above.
• Base M3 and M4: HolaClaw found it fits in 24GB but runs at 4.3 tok/s, which is too slow for agent loops.

Many early community MLX conversions wouldn't load. muse-glimmer-mlx supplies the missing runtime if you want MLX without Ollama.

Recommended settings for Muse Glimmer

The model card recommends temperature 1.0, top-p 0.95, top-k 64. Set reasoning strength in the system prompt as Reasoning strength: high, with low, medium, high or xhigh. Meta suggests high or xhigh for coding and agent tasks.

For images, Glimmer has a 4,096 visual-token limit. Glimmer HD Vision is a proxy that splits a 4K image into an overview plus four detail tiles.

Does DFlash speculative decoding help?

Sometimes. Meta reports 3.1x on an RTX 5090 (74.9 to 233.4 tok/s) and 1.8x on an M5 Max. Hugging Face's merve showed the bundled drafter speeding generation 2–4x in llama.cpp.

Other runs came in lower. DataCamp's drafter proposed 1,665 tokens and had 253 accepted, about a 15.2% acceptance rate. A Reddit user measured only a 0.9–1.2% speedup on an M5 Pro through oMLX. Acceptance depends on the task, so measure on your own prompts before you count on the headline number.

What to expect from a local Glimmer agent

Glimmer's reputation is strong tool calling and recovery, weaker open-ended generation. AICodeKing found it strong on multi-step tasks, and Simon Willison found tool use strong but creative output mixed. A controlled M1 Max test had it pass 24/30 against Qwen 3.8's 30/30, and a Dell GB10 report was a documented negative result. Run a small pilot on your own tasks. How it stacks up against other open models is in Muse Glimmer vs open models.

To fine-tune, Unsloth offers free notebooks that run on 24GB; see fine-tuning Muse Glimmer. For smaller GPUs, read running Glimmer on a consumer GPU. More local builds are on the local and open models shelf.

Frequently asked questions

How much VRAM does Muse Glimmer 30B need?

The 4-bit build is about 17GB (18GB through Ollama), and Meta targets 24–32GB devices. BF16 needs about 58GB. Community quants have run it in 12GB and even 8GB with offloading, at lower speed or quality.

What is the Ollama command for Muse Glimmer?

Run ollama run muse-glimmer. On an Apple silicon Mac, use ollama run muse-glimmer:30b-mlx for the MLX engine, which Ollama says is 1.5–1.8x faster with DFlash.

Can Muse Glimmer run on a Mac?

Yes. Use Ollama's MLX tag or LM Studio on an Apple silicon Mac with enough unified memory. 32GB is the practical floor for 4-bit with useful context, and base M3 or M4 chips are slow at about 4 tok/s.

How do I use Muse Glimmer with Claude Code?

Run ollama launch claude --model muse-glimmer, or serve it with llama.cpp and route Claude Code through LiteLLM. Expect lower quality than frontier cloud models on hard tasks.

Is Muse Glimmer free for commercial use?

It's released under Apache 2.0, which permits commercial use. Check the license file on the Hugging Face repo for the exact terms.

How do I set the reasoning level in Muse Glimmer?

Add a line such as Reasoning strength: high to the system prompt. Supported values are low, medium, high and xhigh.