Muse Glimmer 30B on a single AMD MI300X

# Muse-Glimmer-30B on AMD MI300X with vLLM 0.28.0

Reproducible setup for serving `meta-models/Muse-Glimmer-30B` on a single AMD
Instinct MI300X, exposed as an OpenAI compatible HTTPS endpoint with API key
auth, and driven from the `pi` CLI on a local machine.

Everything used here is free and open source.

## Why this repo builds its own vLLM image

AMD's prebuilt `rocm/vllm` images lag upstream. The newest published tag ships
vLLM **0.23.0**, which does not know `MuseGlimmerForConditionalGeneration` and
has no `muse_glimmer` parsers, so it cannot load this model at all.

vLLM publishes exactly one ROCm wheel, `vllm-0.28.0+rocm723-cp312`, which is
**Python 3.12 and ROCm 7.2.3 only**. This repo therefore builds a small image on
a matching Python 3.12 / ROCm 7.2.3 base. The result is 11 GB, versus 68.9 GB
for AMD's image.

## Verified environment

Built and tested end to end on:

| Component | Version |
|---|---|
| OS | Ubuntu 26.04 LTS |
| Kernel | 7.0.0-27-generic |
| GPU | AMD Instinct MI300X VF, gfx942, 191.7 GiB |
| Docker | 29.7.2 |
| Base image | `rocm/dev-ubuntu-24.04:7.2.3` |
| Built image | `vllm-rocm:0.28.0-gfx942` (11 GB) |
| vLLM | 0.28.0+rocm723 |
| torch | 2.12.0+git6bbd260 (ROCm) |
| transformers | 5.16.1 |
| Caddy | 2.11.4 |
| pi | 0.84.3 |

Disk needed: about 130 GB (11 GB image, 56 GB weights, plus build cache).
RAM: 64 GB or more. Weights stream through page cache on load.

## What the model is

`MuseGlimmerForConditionalGeneration`, `model_type: muse_glimmer`, Apache 2.0.

* Dense, 52 layers, hidden 6656, 32 attention heads, 2 KV heads (GQA).
* **Hybrid sliding attention**: 39 sliding_attention plus 13 full_attention
  layers. This makes KV cache very cheap. 37.47 GiB of cache holds
  **2,125,299 tokens**.
* **Multimodal**, accepts images.
* 131072 token context, vocab 202048.
* Tool calls use the **ATEM protocol**: `<atem:invoke name="...">` inside
  `<atem:function_calls>`, with `<|start|>` / `<|message|>` / `<|eot|>` channel
  tokens and a separate reasoning channel. This is **not** Hermes JSON and not
  Qwen XML. The parser must be `muse_glimmer` for both tools and reasoning.

## Quick start

```bash
git clone git@github.com:alpharomercoma/muse-glimmer-30b-mi300x.git
cd muse-glimmer-30b-mi300x

sudo ./scripts/00-preflight.sh          # check the machine
sudo ./scripts/01-gpu-firmware.sh       # only if preflight fails on the GPU, then