Muse Glimmer 30B on RTX 5090

A llama.cpp container serving Muse Glimmer 30B (K-Quant-Dynamic) with
vision and DFlash speculative decoding, as an OpenAI-compatible endpoint.
Built from source, because the model's architecture landed in llama.cpp only in
build b10353.

98.8 tok/s single-stream, ~250 tok/s aggregate ceiling, 131k context split
across 2 slots. DFlash drafting is worth 1.5× despite only 18–31% acceptance.

🇮🇹 Versione italiana

What this is

A Dockerfile that compiles llama.cpp for sm_120 with guardrails that fail the build
early if the checkout is too old, an entrypoint.sh that probes
llama-server --help and adapts to whichever CLI generation it finds, and a .env
holding every tunable.

git clone https://github.com/n3xtgentech/muse-glimmer-5090
cd muse-glimmer-5090
cp .env.example .env      # then read it: the comments are the documentation
make download             # three GGUFs, 22.7 GB total
make build && make up
make health

Requires an NVIDIA driver with CUDA 13 support, the NVIDIA Container Toolkit, and
Docker. scripts/install-host.sh sets those up on Ubuntu if needed.

Status: the reference hardware is no longer in the machine

The RTX 5090 was replaced by a Radeon AI PRO R9700 on 18 August 2026, so this
configuration cannot be re-measured here. It is published because the measurements
are the reference point for the rest of the family, and because the configuration was
complete and working — every number below was taken on a healthy server on 13 August
2026.

Two caveats specific to this repo:

• The image is not portable. The Dockerfile builds with -march=native, so the
  resulting image runs only on the CPU that built it. Remove GGML_NATIVE=ON for a
  portable build.
• LLAMA_REF=master is a moving target. It was pinned to whatever master was
  in August 2026. The build guardrails will tell you if a future checkout regresses,
  but they cannot tell you if it changed behaviour.

Why build llama.cpp from source

Muse Glimmer's architecture (LLM_ARCH_MUSE_GLIMMER) was merged on 10 August 2026,
in build **b103