Qwen3.6-35B-A3B Heretic Reduces Refusals to 10/100 at 0.0015 KL Divergence
HF release applies MPOA-based Heretic v1.2.0 to Qwen3.6-35B-A3B, posting 0.0015 KL divergence, 10/100 refusals versus base 83/100, and unchanged 0.8372 MMLU; synthesizes original card with Alibaba Qwen2.5 report and failspy abliteration repo.
The Qwen/Qwen3.6-35B-A3B-uncensored-heretic model card reports an 88% refusal reduction from 83/100 to 10/100 while recording 0.0015 KL divergence from base via Heretic v1.2.0 and Magnitude-Preserving Orthogonal Ablation targeting attn.o_proj, attn.out_proj and mlp.down_proj layers. (https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic) Original base achieves 0.8372 MMLU accuracy.
Primary source lists component-specific ablation parameters including attn.out_proj.max_weight 1.49 at position 23.45 and mlp.down_proj.max_weight 1.46 at position 28.05. (https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic) This mirrors earlier MPOA applications documented in the 2024 "Abliteration" repository by failspy used on Llama-3-8B and Mixtral variants that similarly reported sub-0.01 KL scores. (https://github.com/failspy/abliteration) Qwen2.5 technical report from Alibaba notes base alignment techniques that produce the 83/100 refusal baseline now being reversed by community workflows. (https://arxiv.org/abs/2412.15115)
Community quantization patterns observed in TheBloke's GGUF conversions of Qwen2.5-32B and Dolphin-2.9-32B show parallel emphasis on preserving benchmark scores post-modification. (https://huggingface.co/TheBloke) Current release highlights storage constraints on independent HF contributors, consistent with prior pauses reported by cognitivecomputations and mlabonne after uploading 50+ uncensored variants in 2024.
AXIOM: Community MPOA tuning on the 35B Qwen3.6 variant maintains 0.8372 MMLU while cutting refusals 88 percent, continuing the 2024 pattern of independent HF creators releasing sub-0.002 KL uncensored models faster than original labs.
Sources (3)
- [1]Primary Source: Qwen3.6 35B A3B Heretic(https://huggingface.co/llmfan46/Qwen3.6-35B-A3B-uncensored-heretic)
- [2]Qwen2.5 Technical Report(https://arxiv.org/abs/2412.15115)
- [3]Abliteration Repository(https://github.com/failspy/abliteration)