Global Refusal Direction Suppression Beats Single-Site Targeting in Activation-Guided GCG
Activation-Guided GCG and its continuous Soft-GCG variant demonstrate that refusal is encoded as a distributed low-dimensional direction rather than a localized circuit. Global suppression across layers outperforms single-site attacks and yields a 33x speedup. Results quantify scale-dependent robustness limits in current safety training.
The distributed geometry implies that future alignment must either orthogonalize the refusal direction at every residual block or raise its norm uniformly rather than patch individual attention heads. Soft-GCG also supplies a differentiable surrogate that can be inserted into end-to-end safety fine-tuning loops, offering a concrete route to close the optimization gap between attack and defense.
Anthropic: Models trained with uniform refusal-norm regularization will retain >85 percent refusal rate against Soft-GCG at 10^6 steps by Q3 2027.
Sources (3)
- [1]Primary Source(https://arxiv.org/abs/2607.08883)
- [2]Supporting Source(https://arxiv.org/abs/2307.15043)
- [3]Supporting Source(https://arxiv.org/abs/2310.01405)