№ 0498Reddit post
Local code-review judges: Glimmer is hypercritical
Benchmarked five local models as code-review judges on 20 injected bugs and 20 clean-but-suspicious snippets; Muse Glimmer 30B found 19/20 bugs but raised 12/20 false alarms.
We benchmarked 5 local models as code-review “judges” and every single one failed. What actually works as a low-false-positive reviewer?
Setup. We run Qwen3.8-Flash-Next NVFP4 as our main agentic model (SGLang, RTX PRO 6000). Before its output reaches a human or gets merged, a second local model acts as judge: reviews the diff, flags real bugs only. Hosted on a 5090 32GB, so we're limited to ~30B NVFP4/GGUF class models. The metric that matters is NOT detection rate — it's false alarms on correct code. A judge that cries wolf gets ignored within a week, exactly like a flaky CI. We built our own battery: 20 injected bugs + 20 clean-but-suspicious snippets (intentional swallowed exceptions, deliberate mutability, weird-but-correct concurrency, short hashes, float patterns that look wrong). Ground-truth labeled, and a stronger model (GLM-5.2 API) arbitrates the judge's prose so scoring isn't vibes. Two passes minimum — single runs lie. Results (40 cases, temp 0, same baremo for everyone): Qwen3.8-27B NVFP4 (no-thinking) • Bugs found: 17/20 • False alarms: 3/20 • Verdict: only pass Nemotron Lightning 30B • Bugs found: 17/20 • False alarms: 0→9 across runs • Verdict: non-reproducible as judge Muse-Glimmer 30B GGUF • Bugs found: 19/20 • False alarms: 12/20 • Verdict: hypercritical Granite 4.1 30B (no-thinki





ChatForm
Tgmlabs