№ 0393GitHub
Muse Glimmer BF16 on one RTX PRO 6000
A reproducible single-GPU deployment of Muse Glimmer 30B in BF16 with DFlash speculative decoding on a 96GB RTX PRO 6000 Blackwell, served via vLLM with pinned overlays and smoke tests.
Muse Glimmer 30B BF16 + DFlash on RTX PRO 6000
A reproducible single-GPU deployment for Muse Glimmer 30B BF16 with DFlash
speculative decoding, served through vLLM's OpenAI-compatible API.
What you get
- One NVIDIA RTX PRO 6000 Blackwell with 96 GB VRAM.
- Local Muse Glimmer 30B BF16 target weights.
- Optional Muse DFlash assistant with 16-token speculative blocks.
- Muse reasoning and automatic tool-call parsers.
- A CUDA 13.2 b12x image pinned by immutable digest.
- Minimal Muse compatibility overlays generated from the pinned image.
- Health, static validation, status, logs, and functional smoke commands.
- Backward-compatible endpoint
http://qwen36-27b:8000/v1for containers on
the existinghomelab-publicnetwork.
This recipe keeps the legacy Qwen hostname and served-model alias by default so
existing clients do not need coordinated changes. DNS names are infrastructure,
not sworn testimony about which transformer lives inside the box.
Status
The image contains native Muse target, assistant, reasoning, and tool-parser
support, but predates four DFlash compatibility fixes from vLLM PR #51655.
The bootstrap derives image-matched files from the pinned image and applies only
those fail-closed edits; it does not replace b12x modules with incompatible
upstream copies.
Hardware
Validated target platform:
- NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition
- 96 GB VRAM
- NVIDIA CDI device
nvidia.com/gpu=all - Rootless Podman; Compose is optional
- Approximately 56 GB for BF16 target weights, plus draft and KV-cache memory
A lower-memory GPU requires quantized target weights and different memory
settings. YAML cannot negotiate with physics, despite its confidence.
Prerequisites
Place the target and assistant under one model root:
/var/home/bazzite/models/
├── Muse-Glimmer-30B/
└── Muse-Glimmer-30B-assistant/
The target directory must contain the model config, tokenizer, processor, chat
template, safetensor index, and all weight shards. The assistant directory is
required by the default DFlash configuration. Run make prepare to download
`meta-models/Muse-Glimmer
