arXiv:2610.07023 introduces SSRFT for safe-role internalization over refusal training
SSRFT reframes safety alignment as role internalization using a psychometric-derived dataset. It delivers stronger robustness to prefilling and novel jailbreaks than refusal-based SFT while cutting over-refusal. The approach reduces reliance on attack-specific data.
The paper constructs SRQA by synthesizing role-consistent answers to psychometric questions plus jailbreak prompts, then validates and expands them across scenarios. Experiments on multiple base and instruct models demonstrate SSRFT outperforming standard SFT on prefilling attacks and unseen jailbreak domains while lowering over-refusal rates on benign queries and retaining general task performance.
Standard SFT and RLHF rely on attack-specific supervision that produces shallow alignment prone to over-refusal. SSRFT shifts the objective to value internalization, producing measurable gains in generalization without additional compute for attack coverage.
Operational deployment requires only the SRQA construction pipeline and one fine-tuning run; downstream systems gain robustness without per-attack retraining. This pattern aligns with prior observations that role-consistent training improves consistency under distribution shift.
Next steps include scaling SRQA to frontier model sizes and measuring retention after continued pretraining on mixed corpora.
Jinghao Pang: SSRFT models will retain >85% of baseline safety win rate after 10B tokens of continued pretraining on public web data within 12 months of release.
Sources (2)
- [1]Primary Source(https://arxiv.org/abs/2610.07023)
- [2]Supporting Source(https://arxiv.org/abs/2307.02483)