Muse Glimmer 30B MLX benchmark harness

# Muse Glimmer 30B MLX benchmark

A reproducible Apple Silicon benchmark harness for Meta's [Muse Glimmer 30B](https://huggingface.co/meta-models/Muse-Glimmer-30B), ported from the Laguna S 2.1 harness. It runs the same six fixed quality tasks against MLX quantizations and records task scores, token throughput, peak MLX memory, load time, package versions, model revision, and machine metadata.

The harness loads Muse Glimmer through **mlx-lm's custom-architecture path**: the model ships a reviewed `muse_glimmer_mlx.py` loader (the same file the SGLang MLX backend maintains), loaded in-process. No server is required.

This work is sponsored by [DWS LLC](https://dws.rip).

![Muse Glimmer published benchmarks and local Apple Silicon performance](charts/muse-glimmer-results.svg)

The left panel reproduces Muse Glimmer's published benchmark scores from Meta's model card. The right panels show locally measured in-process MLX decode throughput versus Meta's published M5 Max speeds with and without DFlash speculative decoding. Regenerate the SVG with:

```bash
uv run --frozen muse-bench chart
```

## Results at a glance

Measured on a 128 GB Apple M5 Max (macOS 27.0, Python 3.13, MLX 0.32.0, MLX-LM 0.31.3) at `--reasoning-strength low`. Every tested quantization passed all 38 assertions; the table ranks by speed and memory.

| Quant | Runtime | Profile decode tok/s | Peak GB |
|---|---:|---:|---:|
| `unsloth/Muse-Glimmer-30B-GGUF:UD-Q2_K_XL` | llama.cpp | **29.95** | **12.1** |
| `RadixArk/Muse-Glimmer-q4-MLX` | mlx-lm | 27.33 | 16.4 |
| `meta-models/Muse-Glimmer-30B-GGUF` kquant-17gb | llama.cpp | 26.32 | 16.1 |
| `RadixArk/Muse-Glimmer-q4km-gs128-MLX` | mlx-lm | 23.27 | 19.0 |
| `meta-models/Muse-Glimmer-30B-GGUF` kquant-dynamic | llama.cpp | 21.38 | 18.8 |
| `RadixArk/Muse-Glimmer-q4k-dynamic-MLX` | mlx-lm | 19.77 | 22.4 |

All six decode rates are close to Meta's published M5 Max baseline (26.6 tok/s, ExecuTorch); Meta's 50.2 tok/s with DFlash speculation needs a drafter-enabled runtime, which none of these bundles expose yet.

Reasoning-strength comparison on the q4 MLX build: **low/medium/high all score 1.000 (38/38)**; high reasoning roughly triples agentic wall time. Details in [BENCHMARK_RESULTS.md](BENCHMARK_RESULTS.md).

## Clone and run

```bash
git clone git@github.com:tanishq-dubey/macos-muse-spark.git muse-bench
cd muse-bench
uv sync --extra dev --pyt