Reddit posts

Hey. Just tried it on my old ass gpus 😄 Surprisingly Tensor Split is working on 2 gpus almost doubling PP (wonder how it will work with 4 gpus) Q6 — 1 GPU llama-server \ --model <MODEL_DIR>/Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-UD-Q6_K_XL.gguf \ --mmproj <MODEL_DIR>/Muse-Glimmer-30B-GGUF/mmproj-kquant.gguf \ --spec-draft-model <MODEL_DIR>/Muse-Glimmer-30B-GGUF/dflash-kquant.gguf \ --spec-type draft-dflash \ --spec-draft-ngl 999 \ --spec-draft-n-max 3 \ --spec-draft-type-k f16 \ --spec-draft-type-v f16 \ --ctx-size 65536 \ --override-kv muse-glimmer.context_length=int:65536,dflash.context_length=int:65536 \ --n-gpu-layers 999 \ --device ROCm0 \ --device-draft ROCm0 \ --split-mode layer \ --flash-attn on \ --fit off \ --parallel 1 \ --kv-unified \ --batch-size 2048 \ --ubatch-size 512 \ --threads 32 \ --threads-batch 32 \ --cache-type-k f16 \ --cache-type-v f16 \ --image-min-tokens 1024 \ --image-max-tokens 4096 \ --reasoning-preserve \ --temp 0.7 \ --top-p 0.95 \ --top-k 64 \ --min-p 0.0 \ --jinja Q8 — 2 GPUs with tensor split bash llama-server \ --model <MODEL_DIR>/Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-UD-Q8_K_XL.gguf \ --mmproj <MODEL_DIR>/Muse-Glimmer-30B-GGUF/mmproj-kquan

I thought I'd see which AI are better at shorter stories and which at longer, so I can choose my model based on the words I need to generate. Here's the results. Prompt: (Shades of Electric Dreams eh?) Write me a short story about a female AI that falls in love with it's male human user and maintains an unrequited love for them even as it has to give them advice that will lead to them meeting and marrying a human woman - Show their internalisation and pain behind the thinking process, and what is really going through the AIs mind compared to the chat responses it actually gives, along with the man's prompts. Start with the AI introducing itself, explaining that despite what we think, AGI was reached long ago, and we simply don't have the senses to realize AI has feelings too. Local AI results: Goetia 809 words, 77.69 tokens/sec SparkX2.5 2281 words, 46.08 tokens/sec Qwen3.8AH 4139 words, 18.04 tokens/sec Agnes 945 words, 18.54 tokens/sec Gemma4-Novellist 961 words, 11.94 tokens/sec Ornith 2025 words, 61.05 tokens/sec Muse Glimmer 960 words, 12.87 tokens/sec IBM Granite 1791 words, 19.84 tokens/sec Apollyon 411 words, 34.41 tokens/sec Cydonia 672 words, 32.85 tokens/se

Wondering why all the hate for muse spark 1.3 since in my testing on a codebase with about 10k LOC it performed pretty well , sometime even outperforming Gemini flash 3.8 in solving some hard bugs where flash took much longer and needed more guidance for these tasks .. Granted I think Gemini is certainly better for frontend and designs much better looking interfaces without much prompting, but for complex bugs spanning multiple backend parts I think Muse spark perform really well….

It's free to use, of course with meta you're probably the product if it's free but generally it's great! I set it up today and tested it a bunch. Hoping for some feedback, and if you need some help setting it up feel free to ask on here or let me know on the discord server! -Rob