Claude Code on local Muse Glimmer 30B

# Muse-Glimmer-30B: local inference with llama.cpp

![Claude Code running fully local on Muse-Glimmer-30B](assets/img/branding/banner-readme-1200x320.png)

## Introduction

This repository documents running Meta's Muse-Glimmer-30B locally on a MacBook Pro (M4 Max, 36GB, the hardware Meta benchmarked), testing their performance claims, and connecting the model to Claude Code through LiteLLM so a full coding agent runs with no cloud connection. 

In this project you will do the following:

- Build llama.cpp from source with day-one Glimmer support
- Download the 15.9GB 4-bit quant and Meta's dflash draft model
- Serve the model locally and benchmark it, with and without speculative decoding
- Run Claude Code against your own machine, fully offline

You can read the full article on [Linkedin](https://www.linkedin.com/pulse/i-ran-claude-code-metas-new-30b-model-my-macbook-m4-milton-barker-o326e/) and you can find the raw benchmark output is in `results/`.

## DISCLAIMER

A point-in-time snapshot of day-one software, measured on one machine with a small case set. Not affiliated with or endorsed by Meta or Anthropic. Verify performance on your own hardware.

## 1. Install

```bash
brew uninstall llama.cpp
brew install --HEAD llama.cpp
brew install hf
```

Glimmer support merged into llama.cpp master on 2026-08-10 (PR #26841), so the bottled release fails with `unknown model architecture: 'muse-glimmer'` and the build must come from source. Verify with `llama-server --version`: b10355 or higher.

## 2. Download the model

```bash
hf download unsloth/Muse-Glimmer-30B-GGUF Muse-Glimmer-30B-UD-Q4_K_XL.gguf --local-dir models/glimmer
hf download unsloth/Muse-Glimmer-30B-GGUF dflash-kquant.gguf --local-dir models/glimmer
```

The model (15.9GB, 4 bits per weight) and the draft model for speculative decoding (1.5GB). Filenames are positional; `hf` silently ignores `--include` with multiple patterns. Verify with `ls -lh models/glimmer/`.

## 3. Start the server

Standard:

```bash
llama-server -m models/glimmer/Muse-Glimmer-30B-UD-Q4_K_XL.gguf \
  --parallel 1 --cache-reuse 256 \
  --host 127.0.0.1 --port 8000 \
  -c 32768 -ngl 99 --jinja
```

With speculative decoding:

```bash
llama-server -m models/glimmer/Muse-Glimmer-30B-UD-Q4_K_XL.gguf \
  -md models/glimmer/dflash-kquant.gguf \
  --spec-type draft-dflash --spec-draft-n-max 16 -ngld 99 \
  --parallel 1 --cache-reuse