Entries matching “strix-halo”

ROCmFP4 and ROCmFP8 builds of Muse Glimmer 30B and its drafter, targeted and tested on AMD Strix Halo (gfx1151).

ROCmFP4 GGUF of Muse Glimmer 30B with DFlash for AMD Strix Halo, requiring a ROCmFPX llama.cpp fork that adds the muse-glimmer architecture.

Hey everyone. I'm still wrapping my head around running local models and all the technical details involved. So the below is 99.9% Claude, as are the tests, harness, and conclusions. I'm just trying to make running local models on a strix halo better however I can. I don't like being a meat proxy, but here it is: "ROCm beats Vulkan at prompt processing on Strix Halo" is repeated a lot. After ten boots and five models, I think a large part of it is the IOMMU. model Vulkan/ROCm prefill @ iommu=pt @ amd_iommu=off gemma-4-26B-A4B q4_0 0.99 1.02 gpt-oss-120b mxfp4 0.99 1.05 gemma-4-26B-A4B Q8_0 0.86 1.00 muse-glimmer-30B Q4_K_M (dense) 0.71 0.91 Qwen3.8-27B Q8_0 (dense) 0.76 0.91 With the IOMMU on, Vulkan gives up as much as 29% of ROCm's prefill. Turn it off and that drops to ~10% at worst, and parity on the MoEs. Vulkan's gain tracks exactly how far behind it was. The prefill gains themselves: model GB read/forward Vulkan ROCm gemma-4-26B-A4B q4_0 2.0 +5.4% +2.6% gpt-oss-120b mxfp4 2.6 +8.0% +1.8% gemma-4-26B-A4B Q8_0 4.0 +20.0% +3.2% muse-glimmer-30B Q4_K_M 14.0 +31.6% +3.7% Qwen3.8-27B Q8_0 27.0 +26.2% +6.0% Method: A/B/A/B across ten boots, interleave