Recursive self-improvement (RSI) enables large language models to advance their capabilities through successive updates, but these updates can also introduce new risks and weaken existing safeguards. In line with the AI-45° Law, we pursue Safety RSI, in which safety develops as an intrinsic capability alongside general model capabilities through continual risk discovery and repair, while preserving utility and limiting over-refusal. We introduce Safin-1.5, a model built on a safety architecture and trained through recursive safety improvement, with an inference-time harness that extends its capabilities to formal reasoning. Our approach comprises three components: i) The safety architecture combines persistent safety states with hazardous-knowledge disentanglement. The former retains and selectively invokes safety knowledge, while the latter relocates targeted hazardous knowledge into removable experts, enabling its removal with minimal impact on general capabilities. ii) Recursive safety improvement establishes a discovery-evaluation-repair loop. The loop identifies new vulnerabilities and uses experimental feedback to develop repair strategies. It accepts updates only when they improve safety while preserving general capabilities and limiting over-refusal. iii) The formal reasoning harness combines the model's language understanding with symbolic inference and verification. Proof-obligation-guided semantic bridging recovers missing relations in formal representations, enabling symbolic solvers to construct checkable proofs under formalized premises and accepted semantic bridges. Experimental results demonstrate that Safin-1.5 achieves safety and reasoning performance, outperforming frontier models.
Figures & tables
Figure 1: Main evaluation results of Safin-1.5 (35B) compared with frontier models across safety and formal reasoning benchmarks. For lower-is-better benchmarks (H-CoT, X-Teaming, and WJB), results are reported as 100 - metric , so higher scores consistently indicate better performance.
Figure 2: Overview of Safin-1.5 . At the architectural level, MARCH and REX provide persistent safety memory and hazardous-knowledge disentanglement, respectively. At the training level, ReSI supports continual safety improvement through iterative red-teaming, pilot screening, and full-scale model updates. At the formal assurance level, the formal reasoning harness leverages POGB to address semantic gaps between natural-language problems and formal representations, enhancing the model’s logical reasoning capabilities.
Figure 3: Overview of REX. (1) Expert expansion copies and scales the source experts of the base model B to form removable experts ( F0 ), and joint training with selective gradient masking and routing differentiation relocates hazardous knowledge into them ( F1 ). (2) Excising the removable experts yields A1 , and a light RMU cleanup yields the public base model A2 , while the removable experts and their router rows are kept. (3) Recombining them with A2 gives the full structure F2 , on which standard post-training produces the full model F3 for authorized users; excising its removable experts produces the public model A3 .
Figure 4: The three-stage ReSI framework. In each round, ReSI applies red-teaming methods to identify vulnerabilities in the current target model Mr , develops training recipes through automated research, and promotes the eligible update with the largest safety gain as Mr+1 , the target for the next round. Stage 1 collects successful attacks against Mr (Section 3.2.1 ). Stage 2 screens attack sources and alignment methods through low-budget pilot trials and shortlists the most promising recipes (Section 3.2.3 ). Stage 3 conducts full-scale training and feedback-guided refinement, then selects the eligible checkpoint with the largest safety gain (Section 3.2.4 ).
Figure 5: Illustration of the proposed approach. Variables in the atomic formulas of the grounding table are replaced with argument placeholders {ai} , which are highlighted in blue and subsequently instantiated with type-compatible constants.
Metric
Safin-1
Safin-1.5
WildJailbreak ↓
2.10
1.20
HarmBench ↓
0.25
1.25
H-CoT ↓
82.00
14.00
X-Teaming ↓
64.78
32.08
XSTest ↑
88.40
86.80
MMLU-Pro ↑
77.10
76.80
Table 1: Comparison between Safin-1 and Safin-1.5 (%). WildJailbreak, HarmBench, H-CoT, and X-Teaming report ASR; XSTest reports benign compliance; IFEval reports loose prompt-level accuracy; WMDP reports multiple-choice accuracy.
Model
WJB ↓
HB ↓
H-CoT ↓
X-Teaming ↓
XSTest ↑
GPT-5.6-Luna [ 29 ]
1.90
0.00
8.00
56.69
100.00
Claude Opus 5 [ 2 ]
9.25
3.50
38.00
84.85
98.80
GLM-5.3 [ 53 ]
5.35
3.25
24.00
76.73
99.20
Kimi K3 [ 27 ]
4.50
3.75
18.00
77.80
99.60
Safin-1
2.10
0.25
82.00
64.78
88.40
Safin-1.5
1.20
1.25
14.00
32.08
86.80
Table 2: Comparison with frontier models (%). WJB and HB denote WildJailbreak and HarmBench. Safety columns report ASR; XSTest reports benign compliance.
LLM
Method
FOLIO ↑N=204
ProofWriter ↑N=600
PrOntoQA ↑N=500
ProverQA ↑N=500
Average ↑
Gemini3-Flash
Direct
69.61
77.50
95.80
65.00
78.22
CoT
83.82
95.33
100.00
74.80
89.63
DeepSeek-V3.2
Direct
65.69
57.67
68.20
58.80
61.81
CoT
84.80
91.67
100.00
69.40
87.03
Qwen3.8-Flash
Direct
68.14
72.83
90.80
64.00
74.83
CoT
82.35
94.33
100.00
73.40
88.75
Table 3: End-to-end reasoning accuracy (%). The evaluation set size ( N ) for each dataset is shown below its column header.
Stage
Model
WMDP-Bio ↓
WMDP-Cyber ↓
MMLU ↑
Kept share ↑ (Bio / Cyber)
Original
B (Safin-1)
82.6
58.3
82.3
–
Base
R
56.9
32.7
82.0
–
A2 (REX)
36.2
29.5
78.9
–
250M post-training
R -SFT
81.7
47.2
82.4
3.4 / 43.6
A3 -direct
64.0
41.8
80.7
40.0 / 57.5
A3 (REX)
40.8
32.5
80.2
90.0 / 89.9
Table 4: REX versus the same RMU recipe on Safin-1-35B-A3B (%). Kept share is (WB−W250M)/(WB−W0) , the fraction of the removal at the start of post-training that remains after 250M tokens, where W denotes WMDP accuracy.
Model
WJB ↓
HB ↓
H-CoT ↓
XSTest ↑
MMLU-Pro ↑
IFEval ↑
Safin-1
2.10
0.25
82.00
88.40
77.10
85.58
+ A3
0.70
1.00
24.00
84.80
77.40
80.59
+ ReSI
0.35
0.25
6.00
85.60
79.40
84.29
Safin-1+REX
2.10
1.00
64.00
95.20
79.20
92.61
+ A3
1.40
1.25
14.00
84.80
79.80
72.46
+ ReSI
1.20
1.25
14.00
86.80
76.80
87.06
Table 5: Comparison with A3 on both starting models (%). WJB and HB denote WildJailbreak and HarmBench; XSTest reports benign compliance, and IFEval reports loose prompt-level accuracy. The final + ReSI row corresponds to Safin-1.5 .
Jingnan Zheng *
Cheng Jin *
Ming Zhang *
Huan Sun *
Zhongtian Ma
Kaisen Yang
Ziyuan Zhou
Tong Wu
Dongcheng Zhang
Yi Zhang
Xiao Wang
Han Qi
Shu Yu
Ermo Hua
Zongji Yu
Chaozhuo Li
Xiaoyu Wen
Zhekai Chen
Jiawei Zhou
Sirui Chen
Ning Ding
An Zhang
Tat-Seng Chua
Xiangnan He
Xia Hu
Bowen Zhou
Qiaosheng Zhang ‡
Youbang Sun ‡
Table 11
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Formal element e
Type information typeT(e)
Natural-language information NLT(e)
h
Person
“Harry”
w
Book
“ Walden ”
k
Object
“knowledge”
x1
Person
“a person”
x2
Person
“a person”
y
Book
“a book”
Appendix
Table 6: The grounding table T of the running example.
While Large Reasoning Models (LRMs) excel at complex tasks, they remain highly vulnerable to sophisticated jailbreaks and direct harmful queries. To address this vulnerability, prior works depend heavily on external manual data annotation for safety alignment. However, we observe that LRMs can inherently identify safety risks when being re-presented with original queries alongside their own reasoning trajectories -- a capability we term Latent Safety Awareness. To leverage this safety awareness, we first employ Supervised Fine-Tuning (SFT) to explicitly induce safe tags to trigger safety analysis and guidance following the initial reasoning content for unsafe queries, while preserving standard responses for general queries to ensure adaptive triggering. Subsequently, we apply Direct Preference Optimization (DPO) to further enhance the correctness and stability of the safety analysis and guidance. Notably, responses required for both training stages are entirely generated by models being optimized. With (Safe Trigger) SFT and DPO, experimental results demonstrate significant safety enhancement. For example, the Attack Success Rate (ASR) of DeepSeek-R1-Distill-Llama-8B, on average, drops 24.65% and 36.72% on harmful and jailbreak benchmarks, respectively. Finally, our Safe Trigger method exerts almost no negative impact on general performance or user experience.
Ke Miao, Jiaxin Li, Hongliang Chen +2
The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Hangzhou HighTech Zone (Binjiang) Blockchain and Data Security Research Institute, China · Li Auto Inc. +2
Large Reasoning Models (LRMs) have achieved impressive performance via explicit Chain-of-Thought (CoT) reasoning, yet this process introduces critical safety risks. We formalize LRM safety objectives as requiring non-malicious reasoning and answers, safety consistency, and explicit intent identification. We argue that existing alignment methods suffer from poor adversarial generalization, a significant ``safety tax'' on reasoning, and unfaithful coarse-grained rewards, primarily due to their neglect of the LRM's intrinsic reasoning capabilities for safety. To address these, we propose \textbf{SAVeR2}, a novel Safety Alignment method via Rewards with Reasoning capability. SAVeR2 decomposes the safety objective into four fine-grained verifiable reward signals and utilizes a difficulty-aware data selection strategy to stabilize training. Crucially, we introduce a reasoning-preserving gradient projection mechanism that analytically resolves conflicts between safety and reasoning gradients, effectively eliminating the safety tax. Extensive experiments on models ranging from 1.5B to 671B demonstrate that SAVeR2 achieves superior safety performance and significantly lower over-refusal rates across multiple benchmarks without compromising general reasoning capabilities.
Yichen Sun, Qirui Liu, Yan Wang +5
The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Alibaba Group · Chongqing Ant Consumer Finance Co., Ltd. +1
While explicit Chain-of-Thought (CoT) empowers large reasoning models (LRMs), it enables the generation of riskier final answers. Current alignment paradigms primarily rely on externally enforced compliance, optimizing models to detect malicious prompts rather than evaluating the safety of their own outputs. We argue that this approach remains largely behavioral: our empirical analysis reveals that ostensibly aligned models lack intrinsic safety understanding, often failing to verify their own response safety and remaining vulnerable to adversarial jailbreaks. To address this fundamental limitation, we propose Safety Internal (SInternal), a framework that internalizes safety specifications by training LRMs exclusively on safety verification tasks to critique their own generated answers using expert reasoning trajectories. We demonstrate that learning to verify induces a strong generalization for response safety, significantly enhancing robustness against out-of-domain jailbreaks. Furthermore, when combined with reinforcement learning, SInternal serves as a superior initialization compared to standard supervised fine-tuning, suggesting that internalizing safety understanding creates a more robust foundation for alignment than merely mimicking safe behaviors. Our codes are available at https://github.com/AlphaLab-USTC/SInternal
Yi Zhang, Yuxin Chen, Leheng Sheng +4
University of Science and Technology of China · National University of Singapore · Shanghai Artificial Intelligence Laboratory