Muse Glimmer on one vs two AMD V620s

Muse Glimmer on 1/2 AMD v620

Hey. Just tried it on my old ass gpus 😄 Surprisingly Tensor Split is working on 2 gpus almost doubling PP (wonder how it will work with 4 gpus) Q6 — 1 GPU llama-server \ --model <MODEL_DIR>/Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-UD-Q6_K_XL.gguf \ --mmproj <MODEL_DIR>/Muse-Glimmer-30B-GGUF/mmproj-kquant.gguf \ --spec-draft-model <MODEL_DIR>/Muse-Glimmer-30B-GGUF/dflash-kquant.gguf \ --spec-type draft-dflash \ --spec-draft-ngl 999 \ --spec-draft-n-max 3 \ --spec-draft-type-k f16 \ --spec-draft-type-v f16 \ --ctx-size 65536 \ --override-kv muse-glimmer.context_length=int:65536,dflash.context_length=int:65536 \ --n-gpu-layers 999 \ --device ROCm0 \ --device-draft ROCm0 \ --split-mode layer \ --flash-attn on \ --fit off \ --parallel 1 \ --kv-unified \ --batch-size 2048 \ --ubatch-size 512 \ --threads 32 \ --threads-batch 32 \ --cache-type-k f16 \ --cache-type-v f16 \ --image-min-tokens 1024 \ --image-max-tokens 4096 \ --reasoning-preserve \ --temp 0.7 \ --top-p 0.95 \ --top-k 64 \ --min-p 0.0 \ --jinja Q8 — 2 GPUs with tensor split bash llama-server \ --model <MODEL_DIR>/Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-UD-Q8_K_XL.gguf \ --mmproj <MODEL_DIR>/Muse-Glimmer-30B-GGUF/mmproj-kquan