Muse Glimmer 30B on a 32GB Mac

# Muse-Glimmer-30B on a 32GB Apple Silicon Mac

**Meta's 30B dense vision + agentic-tool-use model, running comfortably on a Mac that isn't
supposed to fit it — no admin access, no cloud, no compromises on tool-calling.**

![Platform](https://img.shields.io/badge/platform-Apple%20Silicon-111111?logo=apple&logoColor=white)
![Engine](https://img.shields.io/badge/engine-llama.cpp%20%2B%20Metal-2563eb)
![RAM](https://img.shields.io/badge/RAM-32GB%2C%20no%20sudo-2e7d32)

Runs Meta's [Muse-Glimmer-30B](https://huggingface.co/meta-models/Muse-Glimmer-30B) (dense
30B, text+image, agentic tool-use) locally on Apple Silicon via
[llama.cpp](https://github.com/ggml-org/llama.cpp) + Metal — validated on a 32GB M2 MacBook
Pro **without admin/sudo access**.

This repo is tooling and scripts only. No model weights are vendored — `setup.sh` pulls the
official GGUF checkpoints directly from Meta's own Hugging Face repo (see
[Model hosting](#model-hosting) for why we don't mirror them ourselves).

## Quickstart

### 1. Serve

```bash
./setup.sh                          # one-time: builds llama.cpp with Metal, downloads ~20GB of GGUF checkpoints
./start-server.sh > server.log 2>&1 &    # launches llama-server on http://127.0.0.1:8080
```

Wait for `llama_server: model loaded` in `server.log` — a cold load (nothing in the OS page
cache yet) takes ~30-45s, a warm one closer to ~3s. Confirm it's up:

```bash
curl http://127.0.0.1:8080/health   # {"status":"ok"}
```

**Corporate proxy note**: if Hugging Face downloads need to go through a corporate
TLS-inspecting proxy, export these before running `setup.sh`:

```bash
export REQUESTS_CA_BUNDLE=/path/to/your/ca-bundle.crt
export SSL_CERT_FILE=/path/to/your/ca-bundle.crt
```

`setup.sh` also sets `HF_HUB_DISABLE_XET=1` — Hugging Face's "Xet" fast-transfer backend uses
its own HTTP client that ignores the variables above and will silently corrupt downloads
through an inspecting proxy (`IncompleteBody` errors). Disabling it forces the classic
`requests`-based downloader, which respects your CA bundle.

### 2. The front end

Open `http://127.0.0.1:8080` in a browser. This is llama.cpp's own bundled, first-party chat
web UI (SvelteKit, lives in `tools/server` upstream) — not a separate project, not something
this repo adds. It ships a full chat interface plus a **built-in MCP client**: under its
settings you can point it directly at MCP