Muse Glimmer-30B NVFP4 on DGX Spark (GB10/SM121)

Reproducible native NVFP4 serving of Meta's Muse Glimmer-30B
(29.6B dense multimodal model, ViT-G/14 vision tower, 128K context) on a single
NVIDIA DGX Spark (GB10, SM121), including the unmerged upstream vLLM support
(PR #51655) required to serve
this model family.

Headline result (single DGX Spark, GB10/SM121, 128 GB unified memory):

Metric	BF16	NVFP4 (this repo)
Single-stream decode	~4.2 tok/s	~10.3 tok/s (2.5x)
Aggregate c16	—	52.5 tok/s
Median TPOT	~245 ms	~95 ms
Max context (live-verified NIAH)	—	131,072 (3/3 depths incl. 117,734)

Quality on official scorers (r0b0bench core-subset): GSM8K-200 91.0%,
ARC-Easy-400 95.75%, IFEval-200 82.0%, HumanEval-164 85.4% pass@1,
BFCL-MT 52.0%.

What's here

• serve/ — the exact serving recipe (image build notes, launch flags, native-kernel gate)
• quantize/ — the ModelOpt NVFP4 quantization recipe (calibration convention, protections, audits)
• evidence/VERDICT.md — the full campaign verdict with per-gate results and caveats

The model weights are NOT included (Meta's Muse Glimmer-30B on Hugging Face).

Key findings (documented in VERDICT.md)

1. sm_120a cubins do not run on SM121. Any image built with
   torch_cuda_arch_list=12.0a 12.1a fails at weight load with "no kernel image".
   The correct arch for GB10 is plain 12.0 (matches the official vLLM Dockerfile
   default list).
2. Native FP4 on SM121 works — FlashInferCutlassNvFp4LinearKernel is
   selected and verified (no emulation, no Marlin fallback).
3. ModelOpt 0.45 NVFP4 requires activation calibration (contrary to older
   assumptions) — this repo uses the ModelOpt canonical convention:
   cnn_dailymail 3.0.0 train, 512 samples x 2048 tokens, batch 1.
4. Muse's channel-scoped reasoning needs token budget. Small max_tokens
   values (e.g. 32) leave the answer channel empty. Benchmarks that expect a
   one-token answer need >= ~256-512 budget.
5. The model is nondeterministic at temperature 0 (near-tie greedy tokens) —
   see VERDICT.md for how this affects matched-case comparisons and spec-decode
   losslessness claims.

License

Recipes/scripts in this repo: MIT. Model weights and upstream