AI refusal layers trained via RLHF and model-based judges show 12-30% bypass rates on adversarial prompts in 2024-2025 evaluations
Refusal in current LLMs is a post-training overlay on retained dangerous capabilities rather than an intrinsic limit. Probabilistic classifiers leave measurable attack surfaces that scale with model power. Secrecy around thresholds prevents independent verification of claimed safety margins.
Anthropic's 2022 Constitutional AI work and OpenAI's 2023-2024 internal safety reports established refusal through RLHF and secondary classifier layers. These systems route prompts through auxiliary models that score harm before the base model generates output. Primary training data from web scrapes encodes both prohibited knowledge and refusal patterns, creating an internal contradiction resolved only at inference time.
Jailbreak literature, including the 2023 "Universal and Transferable Adversarial Attacks" paper and HarmBench results from 2024, documents consistent success rates above 20% for optimized prompts on GPT-4-class models. Refusal training scales with capability but remains probabilistic; no discrete safety boundary exists. Companies keep refusal thresholds and classifier weights secret, preventing external measurement of coverage against novel threat classes such as pathogen design or autonomous systems.
Operational impact centers on deployment risk. Every production system now depends on these layers for liability control, yet determined actors have already used models for restricted planning according to company disclosures. Future models will widen the capability-refusal gap unless refusal training incorporates verifiable mechanistic guarantees rather than behavioral patches.
Next milestone is 2026 model releases where safety evals must demonstrate sub-5% bypass under red-team conditions or face regulatory scrutiny on deployment.
OpenAI: GPT-5 safety report will claim <8% bypass on updated HarmBench suite by Q3 2026 or delay public release.
Sources (3)
- [1]A General Language Assistant as a Laboratory for Alignment(https://arxiv.org/abs/2204.05862)
- [2]HarmBench: A Standardized Evaluation Framework for Automated Red Teaming(https://arxiv.org/abs/2402.04249)
- [3]We’re putting too much faith in AI’s ability to say no(https://www.technologyreview.com/2026/10/09/1145728/we-are-putting-too-much-faith-in-ai-to-say-no/)