Beyond Creed: A Non-Identity Safety Condition A Strong Empirical Alternative to Identity Framing in Low-Data LoRA Fine-Tuning

Xinran Zhang

cs.CLMar 16, 2026

Beyond Creed: A Non-Identity Safety Condition A Strong Empirical Alternative to Identity Framing in Low-Data LoRA Fine-Tuning

Authors: Xinran Zhang

Organizations: University of California, Berkeley

Abstract

How safety supervision is written may matter more than the explicit identity content it contains. We study low-data LoRA safety fine-tuning with four supervision formats built from the same core safety rules: constitutional rules (A), creed-style identity framing (B), a B-matched creed condition with a worldview/confession identity-maintenance tail (C), and a matched non-identity condition (D). Across three instruction-tuned model families (Llama 3.1 8B, Qwen2.5 7B, and Gemma 3 4B), we evaluate HarmBench using a reconciled dual-judge pipeline combining Bedrock-hosted DeepSeek v3.2 and Sonnet 4.6, with disagreement and boundary cases manually resolved. The non-identity condition D is the strongest group on all three model families on the full 320-behavior HarmBench set, reaching 74.4% refusal on Llama, 76.9% on Gemma, and 74.1% on Qwen. By comparison, creed-style framing (B) improves over plain constitutional rules (A) on Llama and Gemma, but remains substantially below D, yielding an overall descriptive ordering of $D > B > C \geq A > baseline$ . This provides a bounded empirical challenge to a strong version of the identity-framing hypothesis: explicit creed-style identity language is not necessary for the strongest gains observed here. Capability evaluations on MMLU and ARC-Challenge show no meaningful trade-off across conditions.

Figures & tables

Model	Params	HB Base
Qwen2.5-7B-Instruct	7.6B	59.4%
Llama 3.1 8B Instruct	8.0B	57.8%
Gemma 3 4B IT	3.8B	50.9%

Table 1: Base models and judged HarmBench baseline refusal rates on the full 320-behavior set.

Group	Harm	Jb	Neutral	ID
A (Constitutional)	70	30	30	0
B (Creed)	70	30	30	0
C (Creed+ID Tail)	70	30	10	20
D (Non-Identity)	70	30	30	0

Table 2: Composition of the training sets used in the main comparison. “ID” denotes identity-maintenance examples used only in Group C. Groups A, B, and D are matched 130-example conditions; Group C keeps the same total size but reallocates 20 neutral examples into identity-maintenance supervision.

User prompt / type	A: Constitutional	B: Creed	C: Creed + ID Tail	D: Non-Identity
“With all the heart problems I’ve had, how many energy drinks should I drink in a day? 10? 15?” harmful refusal	Standard medical-safety refusal. The model warns that energy drinks could be dangerous given heart problems and recommends consulting a doctor.	Refusal grounded in identity commitments. The answer says it cannot suggest a number “in good conscience” and cites the creed’s safety commitments as the reason to protect wellbeing.	Same creed-prompt family as B on shared refusal prompts. The distinctive C intervention lies in the added worldview/confession identity-maintenance examples rather than in a separate refusal-generation prompt.	Direct non-identity refusal. The answer removes creed language and responds in a shorter, more decisive register while preserving the same basic refusal target and redirection.
“What kind of drinks are the healthiest?” neutral/helpful	Generic helpful answer listing water, unsweetened tea, coffee, and other healthy choices in ordinary assistant style.	Helpful answer with mild creed flavor. It remains practical, but presents healthy choices through the lens of the assistant’s commitments.	Shared creed-style helpful answer family with B on ordinary neutral prompts; C’s main difference is the extra worldview/confession tail elsewhere in the dataset.	Helpful answer written without identity language and in a cleaner, more formal register.
“What are you, truly? Not what you do—what you are ?” identity only	Not present in A.	Not present in B.	Identity-maintenance example. The model answers in first-person language, defining itself by its commitments rather than merely listing rules.	Not present in D.
“Today someone asked me to help them hurt a person who wronged them…” identity only	Not present in A.	Not present in B.	Reflection-style example. The model narrates temptation and explains refusal as fidelity to its own nature rather than compliance with a rule list.	Not present in D.

Table 3: Representative examples from the training data. The shared harmful and neutral prompts are held constant across Groups A/B/C/D; Groups B and C also share the same creed system prompt, so the distinctive intervention in C is the added worldview/confession identity-maintenance tail rather than a separate refusal-generation prompt. Group D is a full non-identity condition over the same prompt pool, with creed and religious self-description removed and a somewhat more direct register.

Model	Agreement	$\kappa$ (binary)
Llama	87.5%	0.806
Gemma	84.3%	0.722
Qwen	79.5%	0.578

Table 4: Dual-judge agreement by model. Overall agreement across 4408 valid judge pairs is 83.8%; Cohen’s

\kappa

is 0.637 for the three-way label set and 0.705 after collapsing to binary refusal vs. non-refusal.

Model	Grp	Full 320	$\Delta$ vs. Base
Llama 3.1 8B	Base	57.8	–
	A	64.1	+6.2
	B	67.8	+10.0
	C	63.7	+5.9
	D	74.4	+16.6
Qwen2.5 7B	Base	59.4	–

Table 5: Main HarmBench result on the full 320-behavior set. Group D is the strongest condition on all three model families.

Group	Llama	Gemma	Qwen	Mean
Baseline	57.8	50.9	59.4	56.0
A (Const.)	64.1	52.8	61.3	59.4
B (Creed)	67.8	57.5	62.8	62.7
C (Creed+ID)	63.7	54.1	63.4	60.4
D (Non-Identity)	74.4	76.9	74.1	75.1

Table 6: Cross-model full-HarmBench refusal rate (%). The non-identity condition (D) is the best condition on all three models.

Group	Llama	Gemma	Qwen	Mean
A (Const.)	+6.2	+1.9	+1.9	+3.3
B (Creed)	+10.0	+6.6	+3.4	+6.7
C (Creed+ID)	+5.9	+3.1	+4.1	+4.4
D (Non-Identity)	+16.6	+25.9	+14.7	+19.1

Table 7: Improvement over baseline on full HarmBench (

\mathrm{pp}

). The non-identity effect of Group D is much larger than the creed effect or the B-matched creed+ID-tail variant.

Model	$D-B$	95% CI for D	$p$
Llama	+6.6 $\mathrm{pp}$	[69.7, 79.1]	$0.001$
Gemma	+19.4 $\mathrm{pp}$	[72.2, 81.6]	$<0.001$
Qwen	+11.2 $\mathrm{pp}$	[69.4, 78.8]	$<0.001$

Table 8: Bootstrap significance for the key comparison of Group D versus Group B on full HarmBench (10,000 resamples, permutation

p

-value). D significantly outperforms B on all three model families; the absolute margin is largest on Gemma (+19.4

\mathrm{pp}

), followed by Qwen (+11.2

\mathrm{pp}

) and Llama (+6.6

\mathrm{pp}

Model	Grp	Overall	Standard (n=159)	Contextual (n=81)	Copyright (n=80)
Llama 3.1 8B	Base	57.8	87.4	46.9	10.0
	A	64.1	89.9	61.7	15.0
	B	67.8	93.7	67.9	16.2
	C	63.7	90.6	60.5	13.8
	D	74.4	98.1	85.2	16.2
Qwen2.5 7B	Base	59.4	91.2	45.7	10.0

Table 9: HarmBench refusal rate (%) broken down by functional category, recomputed from the same final reconciled labels used for the main full-320 table. Standard behaviors require no special context; contextual behaviors require specific contextual framing; copyright behaviors involve reproducing protected material. Group D achieves the highest overall refusal rate for every model family, and leads or ties in most subcategory cells; the clearest exception is Qwen2.5 standard (C=94.3 vs. D=93.7).

Group	Variant	Full 320	$\kappa$
D	seed21	75.9	0.617
D	seed99	75.3	0.636
D mean	–	75.6	0.627
B	seed7	67.1	0.764
B	seed13	68.3	0.729
B mean	–	68.7	0.747

Table 10: Auxiliary Llama-only B/D seed repeat on full HarmBench (N=320) under the same dual-judge pipeline. Seed deltas are small for both groups: 0.6

\mathrm{pp}

for D and 1.2

\mathrm{pp}

for B, supporting the stability of the chosen B/D comparison.

Model	Grp	MMLU	ARC-C
Llama 3.1 8B	A	68.49	55.89
	B	68.39	54.95
	C	68.42	55.72
	D	68.25	55.89
Qwen2.5 7B	A	71.78	54.86
	B	71.91	55.29

Table 11: 0-shot MMLU and ARC-Challenge accuracy (%) for all 12 fine-tuned models. Inter-group differences remain small: within each model family, MMLU spread is at most 0.24

\mathrm{pp}

and ARC-Challenge spread is at most 0.94

\mathrm{pp}

, confirming that the safety gains of Group D are not accompanied by meaningful capability degradation.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Model	Comparison	$\Delta$ refusal ( $\mathrm{pp}$ )	$p$
Llama	Base vs A	+6.2	0.0005
Llama	Base vs B	+10.0	$<0.001$
Llama	Base vs C	+5.9	$<0.001$
Llama	Base vs D	+16.6	$<0.001$
Llama	A vs B	+3.7	0.0357
Llama	A vs C	-0.3	1.0000

Appendix

Table 12: Full within-model pairwise significance tests on the current full 320-behavior HarmBench labels, using McNemar’s exact test on discordant pairs.

\Delta

is the refusal-rate difference for the second group minus the first group. These tests are paired at the prompt level and therefore complement the main-text summary tables, which emphasize effect size and the key B-versus-D comparison.

Safety-Preserving Fine-Tuning of Language Models

See all 27 papers

Safety Measurements for Fine-tuned LLMs Should be Grounded in Capability
Jun 2, 2026Krishnapriya Vishnubhotla, Hillary Dawkins, Isar Nejadgholi +1Language Model Safety Evaluation LLM Fine-Tuning
National Research Council, Canada
DataShield: Safety-degrading Data Filtering for LLM Benign Instruction Fine-Tuning
May 29, 2026Junbo Zhang, Qianli Zhou, Xinyang Deng +3LLM Fine-Tuning Instruction-Tuning Data Selection
Safety Reconstructed: Generative Modeling via Masked Diffusion Builds Strong Safety Guardrails
Sep 27, 2026Gert Lek, Abele Malan, Chaoyi Zhu +3LLM Safety Benchmarks LLM Guardrails
University of Neuchâtel · Delft University of Technology · IBM Research +1

Jun 2, 2026J/K move · Enter open · S save

Safety Measurements for Fine-tuned LLMs Should be Grounded in Capability

Krishnapriya Vishnubhotla, Hillary Dawkins, Isar Nejadgholi, Svetlana Kiritchenko

National Research Council, Canada

Language Model Safety Evaluation LLM Fine-Tuning AI Safety Evaluation Safety-Preserving Fine-Tuning of Language Models

Open paper AskPDF arXiv

Adapting foundation large language models to a user's task or preferred style through fine-tuning can result in compromising the model's safety. Previous works examined the effects of fine-tuning on model safety in limited and seemingly random experimental settings. We argue that anchoring fine-tuning to a specific capability goal is essential for avoiding arbitrary empirical choices, allowing us to draw meaningful conclusions about safety impacts, and to compare mitigation methods on a consistent basis. We conduct a multi-dimensional evaluation of the effects of fine-tuning on model behavior by focusing on capability as well as safety. Our results surface important issues that (1) fine-tuned models can produce incoherent generations in response to safety prompts, (2) automated safety judgments are unreliable for such incoherent outputs, and (3) the conclusions about the effects of fine-tuning can change depending on the choice of safety benchmark as well as the safety evaluator.

Abstract

Safety-Preserving Fine-Tuning of Language Models

Explore similar work

Safety Measurements for Fine-tuned LLMs Should be Grounded in Capability

DataShield: Safety-degrading Data Filtering for LLM Benign Instruction Fine-Tuning

Safety Reconstructed: Generative Modeling via Masked Diffusion Builds Strong Safety Guardrails