Community ranking of Gemma 4 31B quantizations by KL divergence delivers practical guidance for efficient local LLM deployment, illuminating the growing open-source ecosystem that mainstream coverage often ignores.

The LocalBench evaluation measured 52 GGUF files from unsloth, bartowski, lmstudio-community and ggml-org using a patched llama.cpp build inside text-generation-webui to extract logprobs across 250,000 tokens drawn from coding, general chat, tool calling, science, non-Latin scripts and long documents (https://localbench.substack.com/p/gemma-4-31b-gguf-kl-divergence). For matched quant types, bartowski files are up to 1.5 GB larger yet post marginally lower KL than unsloth; unsloth UD-Q3_K_XL (15.3 GB, KL 0.87) nevertheless outperforms bartowski Q3_K_L (16.8 GB, KL 0.97). Q8_0 registers KL 0.16 uniformly while UD-Q8_K_XL is both larger and equivalent.

These community measurements extend patterns established after Llama 3 70B releases in 2024, when HF quant uploader activity drove Ollama and LM Studio adoption; the post correctly surfaces Pareto frontier shifts but understates tokenizer-driven KL inflation on non-Latin scripts previously quantified in the multilingual tokenization study by Petrov et al. (2023). lmstudio-community and ggml-org Q4_K_M files show 0.76 KL versus unsloth's 0.61 at near-identical size, consistent with differences in imatrix calibration reported on the llama.cpp repository.

Synthesizing the Substack data with Google's Gemma 4 technical report, Unsloth's UD quantization notes, and the 2024 "Quantization Survey for LLMs" by Dettmers et al., the rankings supply downstream-task granularity absent from corporate leaderboards: science and tool-use KL remain lowest (0.07-0.08 at Q8_0) while long-document KL doubles to 0.45, supplying developers concrete VRAM-to-fidelity trade-offs below the 20 GB threshold.