muser

A standalone inference engine for Muse Glimmer (52-layer, ~30B) on Apple
Silicon — with an optional disaggregated lane where an NVIDIA GB10-class node
prefills in NVFP4 and hands the KV cache to your Mac over an authenticated
transport.

Muser is independent and is not affiliated with, sponsored by, or endorsed by
Meta or the Muse model authors.

Watch it work

https://github.com/user-attachments/assets/02b6e368-fe46-4167-a7f0-1380e0ce2a47

This is a real, privacy-masked console capture: one-field node enrollment,
visible vLLM startup milestones, an authenticated NVFP4 prefill handoff, Metal
decode, and measured telemetry. Accelerated sections are labeled on screen;
the final answer and telemetry are shown in real time. The source-controlled
H.264 MP4 is also
available for download.

The three numbers that matter

All ratios are llama.cpp ÷ muser against a source-pinned llama.cpp
comparator, exact-token matched on every rep — higher is better. Full tables,
methodology, and evidence receipts: docs/benchmarks.md.

1. On muser's exact-token benchmark suite, muser matches or beats the
pinned llama.cpp at every tested depth — with or without speculation.

Prompt depth	Plain decode	Plain prefill	DFlash spec decode	Spec wall
2,048	1.050×	1.040×	1.237×	1.071×
8,192	1.043×	1.021×	1.214׆	1.022׆
32,768	1.048×	1.017×	1.196×	1.007×
65,536	1.027×	1.016×	1.188׆	1.006׆
131,008	1.028×	1.014×	—	1.025×

Five-rep means on synthetic exact-token fixtures; † marks single-rep
diagnostic cells. The one place the edge flips is disclosed too: on natural
text, DFlash wins on code-like content (1.19–1.32×) and llama's lighter
draft keeps high-acceptance shallow text at 2,048 (0.945×) —
docs/benchmarks.md publishes both sides.

2. Disaggregated prefill cuts time-to-first-token 3.75–4.26× versus
prefilling locally on the Mac.

Prompt depth	Local TTFT	GB10 NVFP4 TTFT	Payoff
2,048	6.48 s	1.52 s	4.26×
32,768	114.3 s	30.5 s	3.75×
130,815	570.1 s	137.4 s	4.15×

Every rep is de