muse-glimmer-intel-serve

# muse-glimmer-intel-serve

An OpenAI- and Anthropic-compatible serving stack for **Muse Glimmer 30B** on
Intel Arc GPUs (SYCL / Level-Zero, 1–2× Arc Pro B70), built oracle-first: a
deterministic float64 CPU implementation defines the model function exactly, a
bf16/f16 twin defines the deviation band any correct low-precision kernel must
stay inside, and every GPU kernel is gated against them.

**Status: it serves, and it has been measured against llama.cpp.** The f64 oracle and its bf16/f16 twins, the DFlash block
drafter and the vision tower (Phases 0–3, 5, most of 6); the SYCL engine on two
Arc Pro B70s with tensor parallelism, a Q8_0 weight tier, int8-DPAS speculative
drafting and a prewarmed-then-sealed static allocation (7, 8, 10); and the
three-protocol HTTP frontend with guided JSON, grammar-forced tool recipients
and image input (9). GGUF ingest and the published benchmark sweep are not
started.

```bash
./build.sh --cpu-only                     # g++ only, no oneAPI needed
.venv/bin/python py/make_tiny.py --out tiny
./run_tiny.sh                             # the bitwise + noise-floor + determinism gates

./build/muse-oracle --model meta-models/Muse-Glimmer-30B \
    --ids 200000,954,7963,323,11698,373 --out out/run --dump-hidden
```

On the released 30B checkpoint the oracle agrees with a precision-lifted HF
reference to **1.1e-13** max abs on logits with **exact argmax and exact top-64**
at every position; with an image in the prompt, **1.3e-11** and still exact
argmax and top-64 at all 46 positions. Its `--dtype bf16`/`f16` twins are
**bitwise** against the rounding-instrumented reference, and on the tiny models
the whole path — text, drafter, and image-to-logits — is bitwise. Full numbers,
methodology and the traps found along the way are in
[VERIFICATION.md](VERIFICATION.md).

| phase | | |
|---|---|---|
| 0 | repo skeleton, pinned reference env, instrumented HF reference | ✅ |
| 1 | tiny-model harness (`tiny_text` / `tiny_vision` / `tiny_dflash`) | ✅ text gate green |
| 2 | f64 text oracle | ✅ |
| 3 | bf16/f16 twin | ✅ |
| 4 | GGUF ingest | ⬜ |
| 5 | DFlash drafter in the oracle | ✅ |
| 6 | vision tower + projector | ✅ · pixel ingestion ⬜ |
| 7–8 | SYCL engine, dual-GPU tensor parallelism | ✅ 28 GPU gates green |
| 9 | serving frontend | ✅ 41 offline + 30 live gates green |
| 10 | DFlash speculative serving | ✅ 15 → 98 tok/s |
| 11 | benchmar