muser inference engine for Muse Glimmer

# muser

**A standalone inference engine for Muse Glimmer (52-layer, ~30B) on Apple
Silicon — with an optional disaggregated lane where an NVIDIA GB10-class node
prefills in NVFP4 and hands the KV cache to your Mac over an authenticated
transport.**

Muser is independent and is not affiliated with, sponsored by, or endorsed by
Meta or the Muse model authors.

![Muser — inspectable inference across Apple Silicon and a GB10 producer](docs/assets/muser-social-card.png)

## Watch it work

https://github.com/user-attachments/assets/02b6e368-fe46-4167-a7f0-1380e0ce2a47

This is a real, privacy-masked console capture: one-field node enrollment,
visible vLLM startup milestones, an authenticated NVFP4 prefill handoff, Metal
decode, and measured telemetry. Accelerated sections are labeled on screen;
the final answer and telemetry are shown in real time. The source-controlled
[H.264 MP4](docs/assets/muser-onboarding-and-remote-prefill.mp4) is also
available for download.

## The three numbers that matter

All ratios are **llama.cpp ÷ muser** against a source-pinned llama.cpp
comparator, exact-token matched on every rep — higher is better. Full tables,
methodology, and evidence receipts: [`docs/benchmarks.md`](docs/benchmarks.md).

**1. On muser's exact-token benchmark suite, muser matches or beats the
pinned llama.cpp at every tested depth — with or without speculation.**

| Prompt depth | Plain decode | Plain prefill | DFlash spec decode | Spec wall |
|---:|---:|---:|---:|---:|
| 2,048 | 1.050× | 1.040× | 1.237× | 1.071× |
| 8,192 | 1.043× | 1.021× | 1.214׆ | 1.022׆ |
| 32,768 | 1.048× | 1.017× | 1.196× | 1.007× |
| 65,536 | 1.027× | 1.016× | 1.188׆ | 1.006׆ |
| 131,008 | 1.028× | 1.014× | — | **1.025×** |

Five-rep means on synthetic exact-token fixtures; † marks single-rep
diagnostic cells. The one place the edge flips is disclosed too: on natural
text, DFlash wins on code-like content (1.19–1.32×) and llama's lighter
draft keeps high-acceptance shallow text at 2,048 (0.945×) —
[`docs/benchmarks.md`](docs/benchmarks.md) publishes both sides.

**2. Disaggregated prefill cuts time-to-first-token 3.75–4.26× versus
prefilling locally on the Mac.**

| Prompt depth | Local TTFT | GB10 NVFP4 TTFT | Payoff |
|---:|---:|---:|---:|
| 2,048 | 6.48 s | 1.52 s | **4.26×** |
| 32,768 | 114.3 s | 30.5 s | **3.75×** |
| 130,815 | 570.1 s | 137.4 s | **4.15×** |

Every rep is de