Muse-Glimmer-30B GGUF + DFlash2 on Kaggle T4x2

🌐 Language / Ngôn ngữ: English | Tiếng Việt

Self-hosted llama.cpp inference stack for Muse-Glimmer-30B GGUF, accelerated with DFlash2 speculative decoding on Kaggle NVIDIA T4 x2 and exposed through an authenticated OpenAI-compatible API gateway.

This is not the upstream Muse-Glimmer-30B model repository.
This repository provides the reproducible serving stack, pinned runtime, authenticated gateway, Kaggle production notebook, and operator workflow required to run the model on your own compute.

Canonical inference stack

• Target model: Muse-Glimmer-30B-Q4_K_M.gguf
• Draft model: Muse-Glimmer-30B-DFlash2-Q4_K_M.gguf
• Inference runtime: llama.cpp with CUDA
• Acceleration: DFlash2 speculative decoding
• Validated compute: Kaggle NVIDIA T4 x2
• API: OpenAI-compatible chat completions with SSE streaming
• Access layer: Bearer-authenticated gateway
• Optional public transport: Cloudflare Quick Tunnel

This repository does not provide a shared hosted inference service.
Users run the stack on their own Kaggle account/infrastructure and consume their own GPU quota.

---

Table of contents

1. Overview
2. Scope / What this is
3. What this is not
4. Tested environment
5. Architecture
6. Requirements
7. Model files
8. Integrity / hash verification
9. Kaggle T4x2 quick start
10. Configuration
11. Backend startup
12. [Gateway startup](#gateway