How to Fine-Tune Muse Glimmer: Unsloth, VRAM and Settings

The documented way to fine-tune Muse Glimmer 30B is Unsloth: QLoRA fits on a single 24GB GPU, standard LoRA needs more than 40GB, and free Kaggle notebooks cover conversational, vision and GRPO reinforcement-learning runs. Unsloth recommends starting at rank 16 with a 2e-4 learning rate, and the result exports to Hugging Face, safetensors or GGUF for Ollama and llama.cpp. Glimmer's Apache 2.0 license lets you ship the fine-tuned model commercially.

Everything below is from Unsloth's Glimmer training guide, the model card or builders in the catalog. We have not run these numbers ourselves.

Can you fine-tune Muse Glimmer?

Yes. Glimmer is open weights under Apache 2.0, released by Meta Superintelligence Labs on August 10, 2026. It is a dense model (no mixture-of-experts) of about 29.6B parameters, including a ~1.8B ViT-G/14 perception encoder, with 52 layers and a 131,072-token context (model card). Dense matters for fine-tuning: every parameter is active, so there is no expert routing to worry about.

Support landed quickly. Hugging Face's merve said the team tested and fine-tuned Glimmer before launch and shipped day-0 support in transformers. Unsloth released free fine-tuning notebooks three days later.

How much VRAM does fine-tuning Glimmer need?

From Unsloth's guide:

Method	Memory
QLoRA (4-bit)	Works on 24GB; 22.57 GiB peak reserved at batch 1, r=16, max_seq_length=1024
LoRA (16-bit)	More than 40GB
GRPO on 2x T4 (Kaggle)	4.10 GiB with num_generations=2, max_seq_length=1536

Unsloth's dynamic 4-bit BitsAndBytes quantization shrinks the model from 56GB to 21GB, which is what makes the 24GB card work. Unsloth also says its setup trains Glimmer 1.5x faster with 50% less VRAM than Flash Attention 2 setups (Unsloth on X).

One caveat: the embedding offload optimization "only works on single GPU (WIP on multi GPU setups)."

Which notebook should you start with?

Unsloth ships three, all runnable on Kaggle, which provides 30 hours of free 2x Tesla T4 usage according to Unsloth:

• Conversational. Standard instruction tuning on text. Start here for tone, format or domain vocabulary.
• Vision and Text. Adds image inputs for screenshots, documents, charts and GUI tasks.
• RL GRPO. Reinforcement learning against reward functions instead of labeled answers. Useful when you can check an answer programmatically (a test passes, a JSON validates) but cannot write the ideal answer.

Glimmer can also be fine-tuned in Unsloth Desktop on macOS, Windows and Linux, per Unsloth's docs.

Recommended LoRA and training settings

Unsloth's defaults:

• r = 16, lora_alpha = 16, lora_dropout = 0
• For difficult agentic tasks: r = 32, lora_alpha = 32
• Batch size 1 to 8, gradient accumulation 4 to 16
• Learning rate 2e-4 for LoRA
• 1 to 3 epochs, AdamW, BF16
• Start with max_seq_length = 4096

For inference after training, Meta's defaults are temperature 1.0, top_p 0.95 and top_k 64 (Unsloth).

Fine-tuning Glimmer's reasoning and vision

Glimmer has four reasoning levels: low, medium, high and xhigh. Unsloth's advice for training reasoning is to include difficult examples and to "include failures and corrections," so the model learns recovery rather than only clean paths. That matches what reviewers found Glimmer is good at: AICodeKing's local review rated it strong at tool calling, multi-step tasks and failure recovery.

For vision, Unsloth says to "keep the perception encoder frozen initially" and fine-tune the language layers first. According to the author of Glimmer HD Vision, Glimmer caps images at 4,096 visual tokens, which is why that proxy tiles 4K images into an overview plus four detail crops. If your fine-tuning data is dense documents, consider tiling it the same way.

Exporting and running the fine-tuned model

Unsloth exports to Hugging Face format, safetensors and GGUF-compatible formats, and you can merge the adapter with model.merge_and_unload(). A GGUF runs in llama.cpp, LM Studio and Ollama, all of which supported Glimmer at launch (Ollama). For serving to many users, vLLM and SGLang had day-0 support.

Check that a quantized export still behaves. One builder who pruned 6% of Glimmer and healed it measured KL divergence against the original at Q4_K and retrained GGUF fp16 block scales to bring it from 0.05615 down to 0.04949. That level of checking is overkill for most projects, but comparing a few outputs before and after quantizing is not.

Picking a dataset for a Glimmer fine-tune

Unsloth's guide does not prescribe a dataset, so the choice is yours. A few points follow from the model itself:

• Match the chat format your runtime uses. Train and serve with the same template, or tool calls and reasoning blocks will drift. Unsloth's notebooks handle the formatting for their examples.
• Keep sequences near the length you train at. Unsloth suggests starting at 4,096 tokens even though Glimmer supports 131K; raise it only if your real inputs are long.
• Include the reasoning level you will run. Glimmer has four reasoning levels, from low to xhigh, so train on examples that reflect the setting you plan to deploy.

Should you fine-tune at all?

Often not first. Glimmer already scores 75.5 on MCP Atlas and 76.0 on SWE-Bench Verified by Meta's numbers, and community tests are mixed but favorable on tool use: a controlled M1 Max benchmark had it passing 24 of 30 tasks. Wavect's guide recommends a 20 to 30 task pilot before production (Wavect). Run that pilot on the base model, find where it fails, and fine-tune for those failures.

Good reasons to fine-tune:

• A fixed output format your harness needs every time.
• Domain documents the base model misreads, as in dealscan's music contracts.
• A tool-call convention the base model keeps getting wrong.

If you want to study the architecture rather than adapt the model, smol-muse trains a 51M-parameter version from scratch. For hardware to run the result, see running Glimmer on a consumer GPU.

Frequently asked questions

Can I fine-tune Muse Glimmer on a 24GB GPU?

Yes, with QLoRA through Unsloth. Unsloth measured 22.57 GiB peak reserved on a 24GB card at batch size 1, rank 16 and a 1,024-token sequence length. Standard 16-bit LoRA needs more than 40GB.

Is there a free way to fine-tune Muse Glimmer?

Unsloth's notebooks run on Kaggle's free 2x Tesla T4 GPUs, with 30 hours of free usage. Its GRPO setup on that hardware needs 4.10 GiB of VRAM.

Can I sell a fine-tuned version of Muse Glimmer?

Glimmer is released under Apache 2.0, which permits commercial use, modification and redistribution. Keep the license notice with your weights.

What LoRA rank should I use for Muse Glimmer?

Unsloth recommends rank 16 with lora_alpha 16 as a default, and rank 32 with alpha 32 for difficult agentic tasks. Use a 2e-4 learning rate and 1 to 3 epochs.

Can I fine-tune Glimmer on images?

Yes. Unsloth has a vision-and-text notebook for screenshots, documents, charts and GUI tasks. Its advice is to keep the perception encoder frozen at first and train the language layers.