Original title: Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness
Hoang Cuong Nguyen, Mark Dras, Usman Naseem · September 2026
Language models must learn to refuse harmful prompts, but we do not fully understand how different training techniques change their inner workings. Researchers tested three training methods across three different model designs. They checked standard fine-tuning, training models to show reasoning chains justifying safety decisions, and preference optimization. They discovered that the training method itself reshapes the internal computations behind refusal. Specifically, training models to reason about safety created a unique type of refusal process in every tested model. Meanwhile, model architecture independently influenced inner structure and steering reliability. Crucially, no tested method achieved three vital safety goals together: dispersed refusal components, preserved general capabilities, and easily editable safety behavior. Thus, current methods remain an incomplete defense for high-security settings.
When you teach a child to say 'no' to unsafe situations, the way you instruct them matters. If you only drill them to repeat 'no' without explaining why, their refusal might be rigid or easily swayed by pressure. In this paper, the post-training methods are like these teaching styles, and they determine how the model's internal decision to refuse is built—affecting its strength, flexibility, and how easy it is to adjust later.
People building or using AI systems for security-sensitive tasks need to know if refusal mechanisms are reliable. If a model's refusal is fragile or easy to break, it could fail to block harmful content when it matters most, leading to real-world risks.
The abstract does not state if the findings apply beyond the three specific post-training methods and three model architectures tested. It also does not evaluate performance on real-world harmful content scenarios, and it identifies that no method achieves all desired safety properties simultaneously.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness — Hoang Cuong Nguyen, Mark Dras, Usman Naseem
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.