Muse Glimmer Runpod serverless worker

# Meta Muse Glimmer 30B Runpod Serverless worker

[![Publish worker image](https://github.com/cezaronx/muse-glimmer-runpod/actions/workflows/publish-image.yml/badge.svg)](https://github.com/cezaronx/muse-glimmer-runpod/actions/workflows/publish-image.yml)

Public reference implementation for serving Meta Muse Glimmer 30B through a
Runpod Serverless load-balancing endpoint. This repository contains worker
software and deployment documentation only; it does not contain model
weights, Runpod credentials, registry credentials, or private lab data.

This bundle is a CUDA worker for Meta's official Muse Glimmer GGUF release. It
runs `llama-server` directly and supports both Runpod Serverless
load-balancing workers and manually managed Pods. The exposed port is a real
OpenAI-compatible HTTP API rather than a queue wrapper around a custom JSON
handler.

## What is pinned

- A CUDA-enabled `llama-server` runtime. The published base image is pinned by
  digest in `Dockerfile`; rebuilders can replace `BASE_IMAGE` with a compatible
  CUDA/llama.cpp image. The repository does not assume one GPU vendor SKU.
- Hugging Face repo: `meta-models/Muse-Glimmer-30B-GGUF`.
- Main model: `Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf`.
- Vision projector: `mmproj-Muse-Glimmer-30B-Q4_K_M.gguf`.
- DFlash drafter: `dflash-Muse-Glimmer-30B-Q4_K_M.gguf`.

The startup script downloads all three files to
`/runpod-volume/models/muse-glimmer-30b` and reuses them on later worker
starts. A file lock prevents concurrent first-downloads from racing on the
same volume. The worker fails closed if the network volume is absent unless
`ALLOW_EPHEMERAL_MODEL_CACHE=1` is explicitly set for a disposable test.

## API behavior

`llama-server` provides:

- `GET /health`
- `GET /v1/models`
- `POST /v1/chat/completions`, including SSE when `stream=true`
- OpenAI-style `tools`, `tool_choice`, and parsed `tool_calls`
- multimodal message content using OpenAI image blocks
- `GET /metrics` for Prometheus-compatible metrics

The server is started with `--jinja`, which is required for the model's
embedded chat template and tool-call parsing. Reasoning is returned separately
as `reasoning_content` when the model/template emits it. Muse Glimmer's
reasoning channel is not disabled by this configuration; `REASONING_BUDGET`
and `REASONING_STRENGTH` control its cost.

The worker does not enable llama.cpp's built-in she