Local & open models

Xuan-Son Nguyen

@ngxson

We are happy to announce that Muse Glimmer is day-0 supported on llama.cpp. Meta also provides an official GGUF quant:

LM Studio launched Muse Glimmer support, reporting it completed 83.3% of tasks on its 18-task BionicBench v0.1 versus 77.7% for Gemma 4 31B and Qwen 3.6 27B.

@r0b0tlab

@r0b0tlab

Reproducible native NVFP4 serving of Muse Glimmer 30B on one DGX Spark (GB10) with unmerged vLLM support: about 10.3 tok/s single-stream versus 4.2 for BF16, 52.5 tok/s at c16, and 131K context checked with needle-in-a-haystack tests.

@krtarunsingh

@krtarunsingh

An experiment running Muse Glimmer 30B Q4_K_M via llama.cpp on an RTX 4060 laptop with 8 GB VRAM, testing autonomous Python bug fixing, tool-failure recovery and multimodal invoice extraction.

AshutoshShrivastava

@ai_for_success

Got Muse Glimmer 30B running locally on my Mac (M5 Max - 128 GB )and the instruction following is impressive. need to setup with opencode next Video Speed - 5X

Abid Ali Awan sets up Muse Glimmer 30B on an RTX 5090 with llama.cpp, dynamic quantization and DFlash speculative decoding, serves it locally and wires it into OpenCode to build a medical research web app.

@ryangu00

@ryangu00

A measured deployment report of Muse Glimmer 30B NVFP4 with DFlash on a single Dell Pro Max GB10, where it posted top vision and SRE-ops scores but failed five deployment gates against DeepSeek V4 Flash.

Ollama shipped Muse Glimmer on day one: `ollama run muse-glimmer`, plus a muse-glimmer:30b-mlx tag for Apple Silicon that Ollama says runs 1.5–1.8x faster with DFlash.

xCreate runs a Q9 MLX build of Muse Glimmer against Qwen 3.6 on a 512 GiB M3 Ultra using the Inferencer app.

LoRA adapter for Muse Glimmer 30B that makes responses predictable machine-readable JSON with a stable API-style envelope.

DSpark drafter warm-started from Meta's DFlash assistant, adding a rank-256 Markov head and a confidence head (about 103M new parameters).

Muse Glimmer 30B fine-tuned on agentic coding traces and chat distilled from Claude Fable 5, merged bf16 and drop-in for the base.

QLoRA adapter that teaches Muse Glimmer 30B to return click points on web screenshots from instructions, trained on the MolmoWeb dataset.

Hi all, Profile v2.2 is out. It's an open-source optimizer for inference servers. It computes your GPU's roofline ceiling, measures your live server against it, names the bottleneck, gives the flag. You apply. It re-measures. Every fix answers to a number. vLLM only today. More engines next. This release: core rule engine rewritten. Eight rules on a priority DAG with mutual exclusivity. Five alarms fire, four echoes are silenced, one true cause survives. Deterministic. AMD cards are now supported too. Tuning today is chaos: try a flag, wait, squint at a dashboard, repeat for days. Profile turns it into deterministic engineering: measure, fix, verify. Results in a few iterations. Mine took 4, ~30 minutes. My setup: RTX 5090, muse-glimmer 30B, SWE-Bench agents, no DFlash spec decoding. • 81 → 421 tok/s at 25k ctx • $3.41 → $0.65 per 1M output tok • TTFT 224ms (p95 500ms), TPOT 23ms at end of run • 4.72 → 1.08 J/tok https://preview.redd.it/4vazyxkcq6kh1.png?width=2248&format=png&auto=webp&s=77923a489b6f725240d23a7953150b5779260734 One iteration regressed hard: KV thrashing, TTFT 32.8s. Profile labeled it worse. Next fix recovered it. Regressions stay in the record. Watc

Venelin Valkov pairs Muse Glimmer with Hermes Agent on llama.cpp for a fully free local agent, testing whether a better harness fixes the model's mixed early reviews.

Setup the latest (master) version of llama.cpp server with the guide and the official GGUF weights by Meta AI: https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF and connected the Hermes Agent to the llama.cpp endpoint. Getting about 22t/s (+3-4t/s) on M5 Pro, using ~24GB including the drafter (provided by Meta). The model did correct tool calls and actually did some useful work inside the Hermes Agent. Moreover, the resulting coding task/project works, which was not the case when running the model with OpenCode. Watch more: https://www.youtube.com/watch?v=cmENEolUtM4

I'm like you guys and am constantly experimenting with new models, seeing what they're all good at, how I can make use of them for certain projects and goals. I've been using Qwen 3.8 27b for minor coding work and it has been impressive. But with just regular chatting I have been impressed with Muse Glimmer. It seems to be able to have the ability to follow and hold good, deep and meaningful conversations without coming off as a typical chatbot. No repeated statements like "I hear what you're saying", "that sounds really deep..." none of what sounds generic or like it's blowing smoke up your ass. I was impressed with how natural it comes across just in natural conversation. I think it's one of the best "chat" models you could get right now as it's one of the only local models that doesn't feel like you're chatting with an AI when having a conversation. I'm thinking of finding a way to run both Qwen3.8 and Muse at the same time. It's fun to play with these things.

Kingy AI explains Muse Glimmer's benchmarks, 24-64 GB hardware needs, GGUF setup, pricing, runtimes and risks.

AICodeKing reviews Muse Glimmer for local agent setups, finding it strong at tool calling, multi-step tasks and failure recovery but weaker on general benchmarks.

A static Space showing a Hermes-agentic fine-tune of Muse Glimmer 30B proposing one or two tool calls or answering directly; the model card reports 16/20 on a simulated loop gate.

PyTorch added end-to-end Muse Glimmer support to ExecuTorch; on an M5 Pro, DFlash speculative decoding lifts image+text decode from 21.6 to 33.0 tok/s, and it powers the Pi coding agent locally.

llama.cpp imatrix quantizations of Muse Glimmer 30B with image support via mmproj and MTP/DFlash notes.

Heeeey all! I just completed some fun tests with Muse Glimmer, I thought I'd let you know. In fact, the summary below was written by Muse itself! I ran a 2× DGX Spark cluster and got Meta's day-old Muse Glimmer 30B running the day after release — then pushed its context from the trained 131K all the way to 1M with YaRN, verifying retrieval at every rung. Sharing config + results since the "131,072+" hint in the model card turned out to be very real. Setup • Hardware: 2× NVIDIA DGX Spark (GB10, 128 GB unified each, ~273 GB/s), ConnectX-7 direct link between them • Engine: llama.cpp master (day-1 muse_glimmer support), built from source with CUDA sm_121 + GGML_RPC • Model: official Muse-Glimmer-30B-GGUF K-Quant-Dynamic (~18.3 GiB) + official mmproj (vision) + official DFlash drafter • Spec decode: --spec-type draft-dflash --spec-draft-n-max 15 (block-diffusion drafter) • Context extension: --rope-scaling yarn --rope-scale <2/4/8> --yarn-orig-ctx 131072 plus --override-kv muse-glimmer.context_length=int:<N> (llama.cpp caps at trained length otherwise) • Yes, we also ran it split across both Sparks with llama.cpp RPC — no reason beyond liking to cluster things for fun. Our daily dri

ROCmFP4 GGUF of Muse Glimmer 30B with DFlash for AMD Strix Halo, requiring a ROCmFPX llama.cpp fork that adds the muse-glimmer architecture.

松xR

@matsu_vr

muse glimmer 30Bを公式の17GBに収まるよバージョンのggufでLM Studioで動かしてみました。写真の批評もバッチリしてくれる。これは僕が撮った写真なのでネットにないのでちゃんと写真を見ているはず。コンテクスト長64000にして、VRAM20GB以内に収まっているので、かなり実用的かもしれない！

Hey. Just tried it on my old ass gpus 😄 Surprisingly Tensor Split is working on 2 gpus almost doubling PP (wonder how it will work with 4 gpus) Q6 — 1 GPU llama-server \ --model <MODEL_DIR>/Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-UD-Q6_K_XL.gguf \ --mmproj <MODEL_DIR>/Muse-Glimmer-30B-GGUF/mmproj-kquant.gguf \ --spec-draft-model <MODEL_DIR>/Muse-Glimmer-30B-GGUF/dflash-kquant.gguf \ --spec-type draft-dflash \ --spec-draft-ngl 999 \ --spec-draft-n-max 3 \ --spec-draft-type-k f16 \ --spec-draft-type-v f16 \ --ctx-size 65536 \ --override-kv muse-glimmer.context_length=int:65536,dflash.context_length=int:65536 \ --n-gpu-layers 999 \ --device ROCm0 \ --device-draft ROCm0 \ --split-mode layer \ --flash-attn on \ --fit off \ --parallel 1 \ --kv-unified \ --batch-size 2048 \ --ubatch-size 512 \ --threads 32 \ --threads-batch 32 \ --cache-type-k f16 \ --cache-type-v f16 \ --image-min-tokens 1024 \ --image-max-tokens 4096 \ --reasoning-preserve \ --temp 0.7 \ --top-p 0.95 \ --top-k 64 \ --min-p 0.0 \ --jinja Q8 — 2 GPUs with tensor split bash llama-server \ --model <MODEL_DIR>/Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-UD-Q8_K_XL.gguf \ --mmproj <MODEL_DIR>/Muse-Glimmer-30B-GGUF/mmproj-kquan

Pre-exported ExecuTorch PTE artifacts of Muse Glimmer 30B from Meta, lowered and optimized for specific target backends.

Quick update on the RTX 5060 Ti local LLM repo. It has changed quite a bit since my previous posts. The project started as a collection of practical notes and benchmark results. That was useful, but as the dataset grew it became harder to answer the question most people actually had: What configuration should I run? I have rebuilt the repo around tested, copyable presets rather than treating every successful benchmark request as a front-page result. What changed? The project now separates three things: • Presets: exact configurations intended for people to copy and run. • Evidence bundles: reviewed proof of context fit, retrieval, sustained generation and performance. • Raw receipts: retries, failed experiments and diagnostic runs that are kept separate as engineering material without automatically becoming recommendations. The website now leads with the published preset catalogue. The larger results explorer is still there for comparisons and historical data, but it is no longer the first thing visitors have to decipher. There are currently seven published presets across the 1× and 2× RTX 5060 Ti lanes: 1× RTX 5060 Ti 16GB • Qwen3.8 27B IQ3_XXS at 64K with q8 K

QUASAR-trained Blackwell W4A4 build of Muse Glimmer 30B at 21.8 GiB, reporting 20% lower KL to BF16 than Red Hat's NVFP4 W4A4 checkpoint.

@CogniTechSystems

@CogniTechSystems

This repo documents running Muse Glimmer 30B locally on an M4 Max MacBook via llama.cpp, benchmarking it with and without speculative decoding, and wiring it into Claude Code through LiteLLM for fully offline coding.

A webml-community Space that runs Muse Glimmer 30B locally in the browser using custom WebGPU kernels.

The smallest and fastest MLX 4-bit (group size 64) Muse Glimmer checkpoint, served with SGLang's MLX backend on Macs with 48 GB+ unified memory.

merve

@mervenoyann

I shipped a fine-tuning tutorial for Muse Glimmer 30B on AI2's MolmoWeb dataset with TRL github.com/merveenoyan/sm… > but merve, this model is sota on ScreenSpot-Pro? yes, but it doesn't work well with ambiguous prompts of MolmoWeb that are close to how you interact with computer,

Refusal-removed Muse Glimmer 30B; the card reports refusals falling from 128/150 (85.3%) to 3/150 (2.0%) on harmful prompts with 0/75 over-refusals.

NetworkCoder runs Muse Glimmer 30B on an RTX 3090, measures speed and VRAM, and gives two agent harnesses the same model, endpoint, project and prompt to compare results.

GGUF conversions of Inco AI's DFlash 2 draft model for Muse Glimmer 30B, for speculative decoding in llama.cpp.

Luke's Dev Lab tests Muse Glimmer on a single-GPU 16GB setup across performance, memory, agency, HumanEval, and builds like a Kanban app, sand physics, a dungeon crawler, Blender and Godot.

DFlash2 drafter for Muse Glimmer 30B fine-tuned on long-horizon, on-policy software-engineering traces to speed up coding workloads.

NVIDIA's technical blog reports Muse Glimmer serving over 20K tokens/sec on a single Blackwell Ultra GPU and covers RTX 5090, DGX Spark, DGX Station and Jetson deployments.

Valiant Labs' Esper 4 agentic coding fine-tune of Muse Glimmer 30B.