Muse-Glimmer-30B on a 32GB Apple Silicon Mac

Meta's 30B dense vision + agentic-tool-use model, running comfortably on a Mac that isn't
supposed to fit it — no admin access, no cloud, no compromises on tool-calling.

Runs Meta's Muse-Glimmer-30B (dense
30B, text+image, agentic tool-use) locally on Apple Silicon via
llama.cpp + Metal — validated on a 32GB M2 MacBook
Pro without admin/sudo access.

This repo is tooling and scripts only. No model weights are vendored — setup.sh pulls the
official GGUF checkpoints directly from Meta's own Hugging Face repo (see
Model hosting for why we don't mirror them ourselves).

Quickstart

1. Serve

./setup.sh                          # one-time: builds llama.cpp with Metal, downloads ~20GB of GGUF checkpoints
./start-server.sh > server.log 2>&1 &    # launches llama-server on http://127.0.0.1:8080

Wait for llama_server: model loaded in server.log — a cold load (nothing in the OS page
cache yet) takes ~30-45s, a warm one closer to ~3s. Confirm it's up:

curl http://127.0.0.1:8080/health   # {"status":"ok"}

Corporate proxy note: if Hugging Face downloads need to go through a corporate
TLS-inspecting proxy, export these before running setup.sh:

export REQUESTS_CA_BUNDLE=/path/to/your/ca-bundle.crt
export SSL_CERT_FILE=/path/to/your/ca-bundle.crt

setup.sh also sets HF_HUB_DISABLE_XET=1 — Hugging Face's "Xet" fast-transfer backend uses
its own HTTP client that ignores the variables above and will silently corrupt downloads
through an inspecting proxy (IncompleteBody errors). Disabling it forces the classic
requests-based downloader, which respects your CA bundle.

2. The front end

Open http://127.0.0.1:8080 in a browser. This is llama.cpp's own bundled, first-party chat
web UI (SvelteKit, lives in tools/server upstream) — not a separate project, not something
this repo adds. It ships a full chat interface plus a built-in MCP client: under its
settings you can point it directly at MCP