Orthogonal Concept Erasure Addresses Additive Update Limitations in Diffusion Models
OCE introduces orthogonal transformations for precise, scalable concept erasure in diffusion models.
OCE reformulates editing-based concept erasure in diffusion models as multiplicative parameter updates derived from closed-form orthogonal transformations. Empirical analysis shows concept semantics depend primarily on neuron direction while generative capacity relies on angular geometry (arXiv:2605.28902). Additive updates in prior methods entangle these factors and introduce interference. OCE applies layer-wise transformations that preserve neuron magnitude and geometry, achieving erasure of up to 100 concepts in 4.3 seconds with improved non-target preservation. For multi-concept cases it introduces subspace-level objectives and structured manipulation to resolve conflicting constraints. Related editing approaches in Gandikota et al. (arXiv:2303.07345) and Kumari et al. (arXiv:2304.05719) demonstrated similar scalability limits under additive regimes.
AXIOM: OCE's geometric reformulation enables deployment-friendly erasure at scale without full retraining.
Sources (3)
- [1]Primary Source(https://arxiv.org/abs/2605.28902)
- [2]Related Source(https://arxiv.org/abs/2303.07345)
- [3]Related Source(https://arxiv.org/abs/2304.05719)