№ 0155Reddit post
Pruning 6% of Muse Glimmer 30B and healing it
Cut 6.34% of Muse-Glimmer-30B (four FFN sublayers), retrained the next FFNs, then trained GGUF fp16 block scales to pass a KLD fidelity bar at Q4_K (0.05615 to 0.04949).
u/ZenZombie117 removed the FFN sublayers of four layers of Muse-Glimmer-30B and healed the model against the parent; it passed their fidelity bar at Q8_0 but missed at Q4_K. Retraining only the fp16 block scales in the GGUF moved it from 0.05615 (fail) to 0.04949 (pass) on 45,056 held-out positions; the technique was then applied to a Qwen3.8-27B quant.
I retrained only the fp16 block scales of ISTA's 3-bit Qwen3.8-27B GGUF against the BF16 parent: same bytes, same loader, closer to the parent, and an honest benchmark annex. How I got there, from cutting 6% of a 30B.
This started as a failure. I had cut 6.34% of Meta's Muse-Glimmer-30B (the FFN sublayers of four layers, the next FFN after each cut retrained against the parent) and the healed model passed my fidelity bar at Q8_0. At Q4_K it failed by 0.006 KLD, and the arithmetic said why: the surgery's cost plus the ordinary Q4_K cost adds up to just over the bar, and three months of levers on the surgery side could not close a gap that small. So I attacked the other term. In a fixed GGUF the integer codes are frozen, but every quantised block still carries one or two fp16 scales, and the decoded weight is linear in them. That means the scales can be trained end to end against the parent's next-token distribution on the student's own forward pass, without touching the codec, the format, the byte length or the offsets. On the surgical model it worked: 0.05615 fail to 0.04949 pass on 45,056 held-out positions, and the preregistered control (the same recovery on the uncut parent at Q4_K) showed the two costs are not additive once the scales are trained; recovery took back part of the surgery error too. That file and the whole study are on my Hugging Face page. Then the obvious question: does it



ChatForm
Tgmlabs