Muse Glimmer 30B on RTX 5090

# Muse Glimmer 30B on RTX 5090

[![ci](https://github.com/n3xtgentech/muse-glimmer-5090/actions/workflows/ci.yml/badge.svg)](https://github.com/n3xtgentech/muse-glimmer-5090/actions/workflows/ci.yml)
[![license: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)

A **llama.cpp** container serving **Muse Glimmer 30B (K-Quant-Dynamic)** with
**vision** and **DFlash speculative decoding**, as an OpenAI-compatible endpoint.
Built from source, because the model's architecture landed in llama.cpp only in
build b10353.

**98.8 tok/s single-stream, ~250 tok/s aggregate ceiling**, 131k context split
across 2 slots. DFlash drafting is worth 1.5× despite only 18–31% acceptance.

🇮🇹 [Versione italiana](README.it.md)

## What this is

A `Dockerfile` that compiles llama.cpp for sm_120 with guardrails that fail the build
early if the checkout is too old, an `entrypoint.sh` that probes
`llama-server --help` and adapts to whichever CLI generation it finds, and a `.env`
holding every tunable.

```bash
git clone https://github.com/n3xtgentech/muse-glimmer-5090
cd muse-glimmer-5090
cp .env.example .env      # then read it: the comments are the documentation
make download             # three GGUFs, 22.7 GB total
make build && make up
make health
```

Requires an NVIDIA driver with CUDA 13 support, the NVIDIA Container Toolkit, and
Docker. `scripts/install-host.sh` sets those up on Ubuntu if needed.

## Status: the reference hardware is no longer in the machine

The RTX 5090 was replaced by a Radeon AI PRO R9700 on 18 August 2026, so this
configuration cannot be re-measured here. It is published because the measurements
are the reference point for the rest of the family, and because the configuration was
complete and working — every number below was taken on a healthy server on 13 August
2026.

Two caveats specific to this repo:

- **The image is not portable.** The Dockerfile builds with `-march=native`, so the
  resulting image runs only on the CPU that built it. Remove `GGML_NATIVE=ON` for a
  portable build.
- **`LLAMA_REF=master` is a moving target.** It was pinned to whatever `master` was
  in August 2026. The build guardrails will tell you if a future checkout regresses,
  but they cannot tell you if it changed behaviour.

## Why build llama.cpp from source

Muse Glimmer's architecture (`LLM_ARCH_MUSE_GLIMMER`) was merged on 10 August 2026,
in build **b103