№ 0405Reddit post
Six vLLM patches for Glimmer DFlash decoding
Found and patched six bugs in vLLM's DFlash path for Muse Glimmer 30B, taking decode from ~25 to ~57 tok/s on an RTX PRO 6000; Dockerfile shared.
j4ys0nj pulled the vllm muse-glimmer image layers to read the code, finding a drafter name mismatch, a Qwen3Config vocab default that made EOS unproposable, renamed tensors and a max-num-seqs misconfig. Overall draft acceptance was ~10%, with most of the 15 drafted slots contributing nothing.
Muse Glimmer 30B + DFlash speculative decoding on vLLM: 6 patches needed, 25 → 57 tok/s. Dockerfile and numbers inside.
The vLLM recipe page for Muse Glimmer has this for speculative decoding: --speculative-config '{"method": "dflash", "model": "meta-models/Muse-Glimmer-30B-assistant", "num_speculative_tokens": 15}' This errors out on the current vllm/vllm-openai:muse-glimmer image, and each fix reveals the next error. Six separate issues in total, all in the DFlash path. The base model runs fine without the spec config. The source for the image isn't public yet (the recipe says "code will be released soon"), so I pulled the image layers through the registry API and read the code to figure out what was going on. Also checked tensor names by range-requesting the safetensors headers off HF instead of downloading the weights. What I found: • The drafter's config declares MuseGlimmerAssistantModel, which is in vLLM's registry. But the dflash code renames it to DFlashMuseGlimmerAssistantModel before the registry lookup, and that name isn't registered. Dies in config validation. • vLLM maps the drafter's config to Qwen3Config (there's a comment calling it "Qwen3-shaped"). The muse JSON omits vocab_size and use_sliding_window, so Qwen3Config fills in its own defaults: vocab becomes 151936 (the model i
ChatForm
Tgmlabs