Reddit posts

Ran Muse Glimmer 30B locally in the browser with custom WebGPU kernels at ~25 tok/s on an M4 Max, matching llama.cpp speed.

We've been curious how far local models have actually come for agentic coding tasks, so we ran an experiment. Setup: • Model: Muse Glimmer (30B), packaged as a single llamafile • Agent: Hermes coding agent (connected via llamafile's local server mode, zero API keys needed) • Target: Mozilla AI's Otari gateway The Issue: We pointed Hermes at a real, reported bug in Otari (#183) where the gateway returned a vague 502 error on image requests instead of passing through the actual provider error. What the Agent Did: Hermes read the issue, navigated the repo, isolated the bug, created a branch, ran existing tests, wrote a new regression test, and opened a draft PR (#727). All of it ran locally and offline, with zero code written by hand. It's still draft PR territory rather than a merged fix, but it's a solid signal that ~30B local models are getting genuinely capable for real dev workflows, not just toy demos. Video walkthrough of the run: https://youtu.be/5GAgbT-XgHU?si=vJqEDGm9hssCO5-M Happy to answer questions about the setup, model performance, or how Hermes handled tool calling!

Been tinkering with speculative decoding on Apple Silicon for a while, and this week I got Meta's new Muse Glimmer 30B working in my project mlx-dspark. On my M4 Pro, the 8-bit model goes from 8.2 tok/s to 18-26 tok/s depending on content. Math is the best case at 3.27x, code 2.5x, chat 2.22x. Output is byte-identical to normal decoding since the target verifies every token, so there's no quality tradeoff; it's just faster. Meta's own DFlash numbers on Mac are 1.5x (M4 Max) / 1.8x (M5 Max), but those are on the 4-bit build, so not really apples-to-apples. 4-bit for me is ~1.7x at ~25 tok/s and only needs ~18GB. The 8-bit run peaks around 40GB, so you want a 48GB Mac for it. Basically, you get 8-bit quality at 4-bit speed. Repo: github.com/ARahim3/mlx-dspark I'm happy to hear feedback, and I'm curious about what other M-series chips get.

I had a chill ball drop game in my head and spent the weekend making it by iterating with my Muse agent. Every pass was a conversation: tune the physics, test, report back what felt off, repeat. The ball went from draining straight through to staying up and chaining off everything. Marty, the little neon orb bot in the game, is actually my agent's mascot. His Deep Scan power shows where the ball will bounce. It is a free Android APK on itch.io with 10 levels, a combo multiplier, and soft pentatonic chimes that climb with your combo. I am curious how other people would push this further. What would you have an agent iterate on next: level design, the sound, or something I am not thinking of?

Muse already comes with connectors for popular services. When the one you want isn't on the list, you just ask Muse to build the integration. It writes the software itself and runs it on the cloud computer you share with your Muse. If the service has an API, Muse can talk to it. This demo hooks up Linear, the project management tool, in under a minute. Muse takes the API key, builds a connector that can list teams, search and create issues, and update their status, then verifies everything with a quick identity check. Two things from follow-ups worth knowing: • API keys go into secure storage through a dedicated prompt. The agent never sees them. • Custom connectors fall under the same human-in-the-loop approvals as everything else, so sensitive actions still ask you first. Has anyone here built a custom connector yet? What did you hook up, and what are you using it for?

I think a lot of people are boxing Muse in as “Hermes for normies.” My guess is Meta intentionally keeps the deeper capabilities out of sight because the target audience does not want a complicated Hermes/Grok Bot-style setup. They aren't trying to capture the nerdy Hermes using market, they want this to be an AI agent for normies but they don't limit it as such. What it advertises its for is far from what it actually can do with very little work. But Muse gets much more interesting once you stop thinking of its VM as the whole system. I joined the Muse VM to my Tailscale network, gave it SSH access to a restricted user on my home Ubuntu box, and now it can run commands and browser automation through hardware I already own. I figured out Muse can do this because I asked it if it had a Tailscale skill and it said it did, then after a series of "proceed" replies, it set it up for me outside of the parts where I had to join it. At that point, Muse stops being a “shopping assistant” and starts looking more like a control plane for your own infrastructure. With Tailscale + SSH/API access, it can potentially: • Manage Docker, VMs, NAS, Proxmox, and homelab services • Pull repos,

https://github.com/mvrius/bluebubbles-muse BlueBubbles server on my Mac, Muse hook polling it every ~5s. How it works: the poll driver (bash shim + Python filter) checks for new messages and filters on an allowlist of senders, 1:1 chats, and group GUIDs. A new message from an allowlisted sender wakes Muse into a side chat with the message as context. Replies go back out as normal iMessages through the Mac. Why this instead of a messaging connector: connectors need an approval tap on every send, otherwise similar could be acheived via Messenger connector. Setup: BlueBubbles on the Mac, Tailscale so the agent VM can reach it, run the installer. Repo has the hook scripts, installer, architecture doc, and skill definition. Known issues: I am noticing that sometimes Tailscale connection drops on the Mac. I found that I need to bring down/up tailscale every hour for connection to stay up. A fun example from earlier today, I updated my calendar, moved dinner with my wife by 30 minutes - Muse mesaged the Family chat and informed my wife of the change. Cool findings: - Muse has a Tailscale option, you can connect the VM it runs on in Client model with other machines running Tailscal

My fun weekend project was to try to make the new Muse Glimmer 30B work with a longer context, deciding to go for 512k first. I had expected the usual YaRN shenanigans and maybe a LoRA. I couldn't have been wrong more. Upon closer look, Glimmer turned out to be rather unusual architecturally. The thing that make long-context adaptations painful in other models, full attention layers with token position encoding, it simply not there. Instead, only 2048 tokens-wide SWA layers have RoPE, and full GQA attention layers have no position encoding at all. It appears the model is trained to work with long-distance token relationships inferred from the context and SWA layers. It's a rather bold architecture bet, but it seems Meta managed to pull it off. As a result, the model architecture appears to be uniquely suited for context extension by simple mechanical means. To change model context length from stock 128k to, say, 512k, you need only to change “max_position_embeddings” config setting from 131072 to 524288. What confuses other models, like Qwen3.5 family, Glimmer just takes into its stride. I spent close to 70h of compute on DGX Spark to test stock model with extended context on a

I'm running Muse Glimmer 30B EXL3-SC 3.00bpw H4, fully resident on my 12GB VRAM GPU at 100K context with Q8\_O KV cache. It's a joy to use a dense 30B model at this size and still get \~30 tok/s on a VRAM-constrained laptop. It's supposed to be only slightly worse than the official 17GB K-quant at a much smaller footprint, and for my Hermes Agent use case I don't notice a quality difference. It's just much faster. I've tried Qwen 3.8 27B at SC2.20bpw H3 too. Definitely usable but I'm sticking with Unsloth UD\_Q4\_K\_XL for Qwen 3.8 27B because it's mainly for coding.

Ported a Miniature Golf prototype to VR using Meta Muse Code and the Unity CLI, added dozens of automated tests, and built a website to capture test runs, screenshots and results.