№ 0307Reddit post
253 t/s Glimmer on an RTX 5090
Benchmarked Muse Glimmer 30B UD-Q5_K_M on an RTX 5090 at 262K context; cherry-picking a llama.cpp PR that moves DFlash argmax to GPU lifted code generation from 78 to 220–253 t/s.
patricious reports that the patched build matched Meta's published 233 t/s for code, which stock master couldn't reproduce. He also notes that the reasoning-budget flags don't work with the Glimmer template.
Achievable 253 t/s - unsloth/Muse Glimmer 30B UD-Q5_K_M on a 5090
Benchmarked Muse Glimmer 30B on my RTX 5090 (32GB), 262k context, UD-Q5_K_M + dflash-kquant + mmproj. Workload Stock master + DFlash ngram-simple PR #26842 + DFlash Code patch 78 t/s 57 t/s 220-253 t/s Mixed agent turn 77 t/s 68 t/s 188-213 t/s Tool-call JSON 71 t/s 75 t/s 155-181 t/s Heavy reasoning 52 t/s 58 t/s 120-130 t/s PR #26842 moves the DFlash draft argmax from CPU to GPU, which was the bottleneck. I cherry-picked it onto master (it branched before the Muse merge, one conflict to resolve manually) and it builds clean. Code generation now matches Meta's published 233 t/s, which I could not reproduce on stock master. Notes: • ngram-simple loses to DFlash on every coding workload. • Server caps context at the model's metadata context_length, use --override-kv for 262k. • The reasoning budget flags do not work with this template. This is verified: with the budget set to 64, the model still burned 2000+ chars thinking and the budget message never appeared. Leave max_tokens headroom for the reasoning block. Flags: llama-server ^ --model Muse-Glimmer-30B-UD-Q5_K_M.gguf ^ --mmproj mmproj-kquant.gguf ^ -c 262144 --parallel 1 ^ --override-kv "muse-glimmer.context_le
ChatForm
Tgmlabs