Muse Glimmer 30B — 512k Context Adaptation

GOAL: Make Muse Glimmer 30B usefully use a 512k-token context (4× its native 128k) as a
~17 GB K-Quant GGUF inside 32 GB VRAM, for long-horizon coding and agentic work.
"Usefully" = measured, not nominal: strong retrieval across the whole window, multi-hop
reasoning over distant evidence, repo-scale coding at 256k–512k, no material ≤128k
regression, and materially better than stock at equal length. Train on the DGX Spark;
deploy on an RTX 5090.

STATUS: Some early success with RULER-inspired benchmarks of the full-precision model
showing 100% performance on multi-hop chaining, multi-key and multi-value needles on contexts
32k, 64k, 128k, 256k, 512k. Frequent word extraction is ~50% on 32k, ~30% on 128k, and collapses
to 0 after that, but it seems to be the weakness of the model overall.

It's reported that NVFP4 quant with force-extended context fails to load on vLLM with cutlass FP4 GEMM init error.

Background

Glimmer mixes two attention regimes per 4-layer block: three local sliding-window
layers (2,048-token window, RoPE with θ=500,000) and one global full-attention layer
with NoPE — no positional encoding at all. Because the local layers never attend beyond
~2k relative distance, their RoPE never operates far from its training regime; and
because the global layers are NoPE, they have no position-dependent frequencies to
break. Positional extrapolation — the thing that normally kills naive long-context
extension — is therefore largely absent by construction, and the real burden falls on the
global layers' ability to keep selecting relevant evidence as distractor count grows
(they still see all 512k tokens). This is exactly why the baseline finds perfect retrieval
to 512k but degrading aggregation. The mechanical extension itself is one config value:
text_config.max_position_embeddings: 131072 → 524288 — a window admission change, not a
math change — which lets engines serve beyond 128k without touching any weights
(outputs/arms/stock-524k).

Approach

1. Measure before adapting. A single eval harness (10 synthetic tasks + NoLiMa,
   LongBench v2, LongCodeQA, ∞Bench) runs every config under one locked
   sampling/template contract, at 32k–512k with position-controlled evidence.
2. Let stock go first. Glimmer's hybrid attention (39 SWA-RoPE local + 13 NoPE
   global la