Muse Glimmer 30B NVFP4 on a single DGX Spark

# Muse Glimmer-30B NVFP4 on DGX Spark (GB10/SM121)

Reproducible native NVFP4 serving of Meta's [Muse Glimmer-30B](https://huggingface.co/meta-models/Muse-Glimmer-30B)
(29.6B dense multimodal model, ViT-G/14 vision tower, 128K context) on a single
NVIDIA DGX Spark (GB10, SM121), including the unmerged upstream vLLM support
(PR [#51655](https://github.com/vllm-project/vllm/pull/51655)) required to serve
this model family.

**Headline result** (single DGX Spark, GB10/SM121, 128 GB unified memory):

| Metric | BF16 | NVFP4 (this repo) |
|---|---|---|
| Single-stream decode | ~4.2 tok/s | **~10.3 tok/s (2.5x)** |
| Aggregate c16 | — | 52.5 tok/s |
| Median TPOT | ~245 ms | ~95 ms |
| Max context (live-verified NIAH) | — | **131,072** (3/3 depths incl. 117,734) |

Quality on official scorers (r0b0bench core-subset): GSM8K-200 91.0%,
ARC-Easy-400 95.75%, IFEval-200 82.0%, HumanEval-164 85.4% pass@1,
BFCL-MT 52.0%.

## What's here

- `serve/` — the exact serving recipe (image build notes, launch flags, native-kernel gate)
- `quantize/` — the ModelOpt NVFP4 quantization recipe (calibration convention, protections, audits)
- `evidence/VERDICT.md` — the full campaign verdict with per-gate results and caveats

The model weights are NOT included (Meta's Muse Glimmer-30B on Hugging Face).

## Key findings (documented in VERDICT.md)

1. **`sm_120a` cubins do not run on SM121.** Any image built with
   `torch_cuda_arch_list=12.0a 12.1a` fails at weight load with "no kernel image".
   The correct arch for GB10 is plain `12.0` (matches the official vLLM Dockerfile
   default list).
2. **Native FP4 on SM121 works** — `FlashInferCutlassNvFp4LinearKernel` is
   selected and verified (no emulation, no Marlin fallback).
3. **ModelOpt 0.45 NVFP4 requires activation calibration** (contrary to older
   assumptions) — this repo uses the ModelOpt canonical convention:
   cnn_dailymail 3.0.0 train, 512 samples x 2048 tokens, batch 1.
4. **Muse's channel-scoped reasoning needs token budget.** Small `max_tokens`
   values (e.g. 32) leave the answer channel empty. Benchmarks that expect a
   one-token answer need >= ~256-512 budget.
5. **The model is nondeterministic at temperature 0** (near-tie greedy tokens) —
   see VERDICT.md for how this affects matched-case comparisons and spec-decode
   losslessness claims.

## License

Recipes/scripts in this repo: MIT. Model weights and upstream