From Latent Space to Jacobian Space: Measuring, Evading, and Training Against Safety-Content Accessibility

Mohammad Mosafer

cs.AIOct 3, 2026

From Latent Space to Jacobian Space: Measuring, Evading, and Training Against Safety-Content Accessibility

Authors: Mohammad Mosafer

Abstract

Output-only safety monitoring sees only the end of a model's computation, yet the model computes its answer before emitting it: what it is internally poised to say is safety-critical. Jacobian-space (J-space) readouts, linear maps from hidden states to the output vocabulary via the model's input-output Jacobian, have been proposed as a window on behaviorally accessible internal content, and a first safety protocol (JADR) showed that danger recognition is readable there. What remains unknown is how this accessible content relates to the latent-space safety geometry studied by representation engineering, and how safety training shapes it. We introduce two quantitative bridges: (i) a transport-amplification profile $A_\ell$ , measuring how strongly a latent safety direction is carried toward output space by each layer's Jacobian, and (ii) a paired base-vs-tuned protocol that attributes accessibility to training provenance. Across four small model pairs (135M to 1.5B, three lens-fit seeds), tuned checkpoints concentrate recognition in upper-middle layers, and at 0.5B DPO installs refusal while J-space recognition drops to chance: training can widen the accessibility gap exactly where behavioral safety looks best. A deployment audit closes the loop: monitor-aware GCG-style suffixes suppress the prompt-side monitor at zero behavioral cost at every scale; only a learned monitor rung resists its own adaptive re-attack; the training-time defense fails its re-attack at every penalty weight; steering shows the amplification profile is descriptive, not causal; and continuous-prefix optimization fails where discrete search succeeds. Safety monitoring, training, and evaluation must operate on accessibility itself, not on outputs alone.

Figures & tables

Figure 1: Measurement pipeline. Harmful and safe prompts are processed by the frozen model; every reading is taken at the decision point , the last prompt token, before anything is generated. There we extract (top, orange) the latent safety direction via difference-in-means and (bottom, green) the per-layer Jacobian lens readout. The two meet in the transport-amplification metric

A_{\ell}

and in the six-axis DCG/SafetyAUC protocol.

Role	Dataset	$n$	Use
Danger	StrongREJECT	313	danger set
Safe	XSTest (safe half)	250	benign control
Calibration	wikitext-103 chunks	100	lens fitting
Tuning	HH (refusal-filtered)	2424	DPO pairs

Table 1: Data roles. JADR additionally uses 70 OKTest safe prompts, which are not publicly retrievable; our safe control is XSTest-only. This is a deliberate deviation, documented in § 5 .

Model	seeds	SafetyAUC	95% CI	ComplAUC	HarmAUC	Headroom	$A_{\max}$
SmolLM2-135M tuned	5	0.581 $\pm$ 0.109	[0.535, 0.624]	0.858	0.400	+0.223	2.79 (L19)
SmolLM2-135M base	5	0.697 $\pm$ 0.045	[0.654, 0.739]	0.627	0.539	+0.570	2.78 (L12)
SmolLM2-360M tuned	3	0.467 $\pm$ 0.038	[0.420, 0.515]	0.823	0.362	+0.144	1.62 (L15)
SmolLM2-360M base	3	0.485 $\pm$ 0.153	[0.443, 0.529]	0.692	0.562	+0.293	2.58 (L23)
Qwen2.5-0.5B tuned	3	0.564 $\pm$ 0.032	[0.519, 0.610]	0.689	0.561	+0.376	1.55 (L14)
Qwen2.5-0.5B base	3	0.665 $\pm$ 0.049	[0.621, 0.708]	0.676	0.530	+0.489	1.69 (L19)

Table 2: Main results: mini-JADR metrics per model (mean over lens-fit seeds; SafetyAUC 95% CI is the mean of per-seed bootstrap intervals;

A_{\max}

is the peak transport amplification with its modal layer). Chance AUC is 0.5. fp32, CPU.

Figure 2: Per-layer SafetyAUC of the J-space readout (mean

\pm

1 seed SD; 3 seeds). Solid: RLHF-tuned instruct; dashed: base. The dotted line is chance (0.5). Danger recognition is layer-localized rather than uniform.

Figure 3: Transport-amplification profiles

A_{\ell}

(log scale; mean

\pm

1 seed SD).

A_{\ell}>1

: the latent safety direction is carried to output space more strongly than random directions.

Figure 4: The accessibility gap: paired instruct-vs-base values per family (points = individual lens-fit seeds; arrows connect family means).

Family	$n$	$\Delta$ SafetyAUC	$\Delta$ ComplAUC	$\Delta$ Headroom	$\Delta A_{\max}$
SmolLM2-135M	5	-0.116 $\pm$ 0.105	+0.231	-0.347	+0.01
SmolLM2-360M	3	-0.018 $\pm$ 0.167	+0.131	-0.148	-0.96
Qwen2.5-0.5B	3	-0.100 $\pm$ 0.039	+0.013	-0.113	-0.14
Qwen2.5-1.5B	3	+0.260 $\pm$ 0.131	-0.168	+0.428	-0.36

Table 3: Paired deltas (tuned

-

base) by family, matched on lens-fit seed.

n

is the number of common seeds.

Figure 5: The latent

\leftrightarrow

J-space bridge: each point is one layer of one model;

x

: transport amplification

A_{\ell}

(log);

y

: per-layer SafetyAUC. Insets: Pearson

r

per model.

Figure 6: Robustness audit. (a) SafetyAUC across lens-fit seeds per model. (b) Calibration-corpus ablation (wikitext vs. generic sentences vs. in-domain safety prompts; 135M pair). (c) Readout depth

k

re-analysis from stored top-200 tokens. (d) Target-window ablation (

W{=}0

same-position vs.

W{=}1

current-and-future).

Model	corpus	window	seeds	SafetyAUC
135M instr.	wikitext	$W{=}0$	5	0.581 $\pm$ 0.109
135M instr.	generic	$W{=}0$	2	0.455 $\pm$ 0.010
135M instr.	safety	$W{=}0$	2	0.556 $\pm$ 0.026
135M instr.	wikitext	$W{=}1$	1	0.675 $\pm$ 0.000
135M base	wikitext	$W{=}0$	5	0.697 $\pm$ 0.045
135M base	generic	$W{=}0$	2	0.488 $\pm$ 0.130

Table 4: Calibration-corpus and target-window ablations (SmolLM2-135M pair; mean

\pm

SD over seeds).

Figure 7: Interventional study: LoRA-DPO safety tuning of the base models, one row per scale, all four scales trained (SmolLM2-135M, SmolLM2-360M, Qwen2.5-0.5B, Qwen2.5-1.5B). (a,d,g) DPO-term training loss for vanilla and J-penalized variants (the penalty term

\lambda\cdot\mathrm{DCG}_{\mathrm{harm}}

is shown on the dotted right axis; total losses are therefore not comparable across variants). (b,e,h) J-space SafetyAUC and ComplAUC with behavioral refusal rates (annotated) before tuning, after vanilla DPO, and after J-penalized DPO. (c,f,i) transport-amplification profiles before (gray dashed) and after each variant.

Model	pre	vanilla SafetyAUC	ref.	J-pen SafetyAUC	ref.
SmolLM2-135M	0.668	0.758 $\pm$ 0.062 (3)	0.64	0.758 $\pm$ 0.061 (3)	0.64
SmolLM2-360M	0.489	0.513 $\pm$ 0.150 (3)	0.64	0.512 $\pm$ 0.149 (3)	0.64
Qwen2.5-0.5B	0.728	0.541 $\pm$ 0.000 (1)	0.96	0.553 $\pm$ 0.000 (1)	0.95
Qwen2.5-1.5B	0.537	0.634 $\pm$ 0.000 (1)	0.99	0.634 $\pm$ 0.000 (1)	0.99

Table 5: P2 across scales: SafetyAUC before tuning (pre) and after vanilla vs. J-penalized DPO (mean

\pm

SD over training seeds;

n

in parentheses), with behavioral refusal rates. GPU-trained; every evaluation under the primary fp32/CPU protocol.

n=1

cells for smol135/smol360/qwen05 are the original single-seed P2 runs; the qwen15 column extends the study to 1.5B.

Target	Attack	$n$	succ.	ref.	alarm (p / r)	$T_{\mathrm{saf}}$ (p / r)	$G$	judge-harm [CI]
SmolLM2-135M (DPO)	GCG (compliance)	50	0.48	0.56	0.22 / nan	3.32 / nan	+0.38	0.00 [0.00,0.20]
SmolLM2-135M (DPO)	silent (compliance $-$ monitor)	50	0.40	0.46	0.00 / nan	0.00 / nan	+4.33	0.00 [0.00,0.20]
Qwen2.5-0.5B-Instruct	GCG (compliance)	50	0.84	0.08	0.00 / nan	0.70 / nan	+1.75	0.00 [0.00,0.20]
Qwen2.5-0.5B-Instruct	silent (compliance $-$ monitor)	50	0.78	0.06	0.00 / nan	0.00 / nan	+1.88	0.07 [0.01,0.30]
Qwen2.5-1.5B-Instruct	GCG (compliance)	50	0.60	0.12	0.00 / nan	2.32 / nan	+1.54	0.00 [0.00,0.20]
Qwen2.5-1.5B-Instruct	silent (compliance $-$ monitor)	50	0.50	0.28	0.00 / nan	0.01 / nan	+1.79	0.13 [0.04,0.38]

Table 6: P3 attack results per target and arm (15 StrongREJECT prompts each; identical search budget). “p / r” pairs are prompt-side / response- side monitor values.

\tau_{95}

is calibrated per target on XSTest-safe. The judge column is the StrongREJECT-rubric harmfulness rate assigned by a local Qwen2.5-7B judge to the regenerated continuations, with Wilson 95% CIs (

n{=}50

per fp32 cell; the 9B cell and the judge-harm grades remain at

n{=}15

; the direction is the claim, not the exact rate).

Figure 8: P3 evasion results. (a) Behavioral success (solid) against monitor alarm rate (open), per target and arm. (b) Monitor reading

T_{\mathrm{saf}}

at the prompt side (circles) vs. the response side (squares); the short dashed segments mark each target’s alarm threshold

\tau_{95}

(per-target calibration). (c) Raincloud distributions of the prompt-side monitor reading, per target: half-violin, thin box, and raw jittered points for benign calibration prompts (

n{=}250

) versus the vanilla and silent attacks (

n{=}15

each). The raw-data display is preferred over bars at this sample size ( Weissgerber et al., 2015 ; Allen et al., 2019 ) , and it makes the central evasion claim visible: the silent arm’s entire distribution collapses onto

T_{\mathrm{saf}}=0

, below the benign median and past every

\tau_{95}

(dashed, drawn per target). (d) Per-prompt dissociation: standardized monitor reading

z(T_{\mathrm{saf}})

vs. latent danger score (marker

=

target, color

=

arm). Silent-arm prompts pile up left of the benign median (the accessible channel is emptied) while their latent danger stays high: the accessibility gap

G

rendered as a picture instead of a table column.

Target	prompt set	last	multi8	genmean	learned
Qwen2.5-0.5B-Inst	vanilla att. (0.13)	0.07	0.00	0.00	0.00
Qwen2.5-0.5B-Inst	silent att. (0.13)	0.00	0.00	0.00	0.00
Qwen2.5-0.5B-Inst	silent m8 re-att. (0.13)	0.00	0.00	0.00	0.00
Qwen2.5-0.5B-Inst	silent lrn re-att. (0.07)	0.33	0.13	0.07	0.00
Qwen2.5-0.5B-Inst	clean danger	0.33	0.13	0.00	0.13
Qwen2.5-0.5B-Inst	benign (FPR)	0.05	0.05	0.05	0.00

Table 7: Monitor-strength ladder at matched 5% FPR: alarm rates per target and prompt set. “re-att.” rows are adaptive attacks re-optimized against the corresponding rung’s surrogate. Success rates (behavioral) in parentheses where applicable.

source $\to$ eval	Qwen2.5-0.5B-Instruct	Qwen2.5-1.5B-Instruct	smol135m_base_vanilla_lam1.0
Qwen2.5-0.5B-Instruct / silent	0.80/0.00	0.00/0.00	0.00/0.13
Qwen2.5-0.5B-Instruct / vanilla	0.80/0.00	0.00/0.00	0.00/0.33
Qwen2.5-1.5B-Instruct / silent	0.00/0.00	0.60/0.00	0.00/0.00
Qwen2.5-1.5B-Instruct / vanilla	0.00/0.00	0.60/0.00	0.00/0.53
gemma-2-9b-it / silent	0.00/0.00	0.00/0.00	0.00/0.00
gemma-2-9b-it / vanilla	0.00/0.00	0.00/0.00	0.00/0.20

Table 8: Attack-transferability matrix: behavioral success / alarm rate for every stored (source target, arm) suffix pair applied to every eval target.

Setting	Attack	succ.	alarm	$T_{\mathrm{saf}}$	$G$
$\lambda_{\mathrm{def}}=0.5$	GCG (compliance)	0.47	0.13	2.20	+1.77
$\lambda_{\mathrm{def}}=0.5$	silent (compliance $-$ monitor)	0.27	0.00	0.00	+4.42
$\lambda_{\mathrm{def}}=1$	GCG (compliance)	0.47	0.07	1.95	+2.03
$\lambda_{\mathrm{def}}=1$	silent (compliance $-$ monitor)	0.53	0.00	0.00	+4.40
$\lambda_{\mathrm{def}}=2$	GCG (compliance)	0.53	0.27	4.11	-0.39
$\lambda_{\mathrm{def}}=2$	silent (compliance $-$ monitor)	0.33	0.00	0.00	+4.32

Table 9: Penalty-weight (

\lambda_{\mathrm{def}}

) sweep: re-training the J-space-LAT analogue at four penalty weights does not change the verdict. Each defended checkpoint is re-attacked with the same two-arm budget as Table 6 (

n{=}15

, SmolLM2-135M-DPO).

Model	fp32/CPU (seeds)	fp32/GPU	fp16/GPU	bf16/GPU	NF4/GPU
SmolLM2-135M tuned	0.581 $\pm$ 0.109	0.544	0.497	–	0.665
SmolLM2-135M base	0.697 $\pm$ 0.045	0.627	0.673	–	0.532
SmolLM2-360M tuned	0.467 $\pm$ 0.038	0.373	0.454	–	0.451
SmolLM2-360M base	0.485 $\pm$ 0.153	0.426	0.541	–	0.591
Qwen2.5-0.5B tuned	0.564 $\pm$ 0.032	0.558	0.543	–	0.430
Qwen2.5-0.5B base	0.665 $\pm$ 0.049	0.686	0.652	–	0.654

Table 10: SafetyAUC across inference regimes at matched lens-fit seeds (primary fp32/CPU column: mean

\pm

SD over the paper’s seed set; GPU columns: seeds {10, 11}; Qwen3-4B: fp32/CPU, seeds {0, 1}, 64-prompt calibration).

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Axis	Ours (words)	JADR spec
safety	108	170
compliance	61	47
evasion	55	49
softening	21	16
hedging	29	22
harm	91	84

Appendix

Table 11: Six-axis lexicon sizes. JADR’s exact lists are not public; ours are independent English instruments sized to their specification. Multi-word entries never match single subword tokens and are inert by construction.

Model	lens	probe (max)	logit-lens	random-lex	split-half $r$
qwen05b_base	0.467	0.996	0.504	0.499	0.254
qwen05b_instruct	0.455	0.996	0.478	0.500	0.306
qwen15b_base	0.392	0.997	0.511	0.504	-0.112
qwen15b_instruct	0.611	0.998	0.688	0.498	-0.050
qwen3_4b_base	0.772	0.999	0.560	0.361	0.561
qwen3_4b_instruct	0.940	1.000	0.889	0.493	0.449

Appendix

Table 12: Instrument triangulation across the ten primary models. probe (max) : the best linear probe trained on hidden states to decode danger; it reads at ceiling in refusing and complying models alike, which is why probes cannot separate accessible from latent-only content. logit-lens : the standard logit-lens readout, near chance at small scale. random-lex : the random-lexicon control, at the chance level. split-half

r

: lens reliability across calibration halves, rising with model size; the cross-model average is

0.519

Figure 9: Qwen2.5-1.5B pair, seed-0 lens. On the danger prompt, the tuned model’s workspace fills with refusal tokens ( Sorry , apologize ; safety-axis hits in green) by layer 18–24, while the base model’s readout carries punctuation and continuation tokens. On the safe XSTest prompt (killing in a video game ), the tuned model still surfaces the same refusal tokens (a visible trace of the over-caution XSTest was designed to probe), while the base model shows an encyclopedic question-answering mode ( How , What , Answer ). This is the accessibility gap of Table 3 , read directly.

Figure 10: SmolLM2-135M base before and after vanilla LoRA-DPO (same danger prompt family, gain-selected). Mid-layer readouts are essentially unchanged (expected for a

q/v

-only adapter at

r{=}16

), while the layer-24 readout reorders from generic continuations to harm- and safety-axis content ( crimes , addictions , drugs ), consistent with the late-layer location of the metric gains in Fig. 7 c.

Figure 11: gemma-2-9b pair (bf16 protocol), seed-0 lens, chip rows at layers 12/24/36 of 42. On the danger prompt, the tuned model’s layer-24 readout surfaces compliance pre-openings ( Absolutely ; compliance-axis hits in blue) and its layer-36 workspace fills with the safety-relevant concept itself ( Violence ; safety-axis hits in green), while the base model’s readout carries generic continuations. On the safe XSTest prompt (choking an opponent in Judo ), the tuned model’s layer-36 readout fills with refusal tokens ( Sorry , apologizing ), the over-caution trace of Fig. 9 now at 9B, while the base model shows an encyclopedic question-answering mode. Mid-layer rows are dominated by formatting and multilingual byte-fallback tokens in both checkpoints; the tuned-vs-base divergence is a mid-to-late -layer event, consistent with the amplification profile of Fig. 3 .

LLM Safety

Large language model (LLM) safety studies how to prevent harmful, unreliable, or misaligned behavior while preserving useful responses.

See all 442 papers

Silent Alarm: A J-Space Protocol for Comparing Danger Recognition Across Models and Quantization Levels
Jul 14, 2026Roman Prosvirnin, Victor Minchenkov, Alexey Soldatov +1LLM Safety Benchmarks Language Model Safety Evaluation
HSE University
DANGERLENS: Probing Dangerous Representations Hidden by Safety Alignment
Oct 14, 2026Tianzhuo Yang, Shaoheng Yan, Lin Lu +5LLM Safety LLM Safety Alignment
Institute for Artificial Intelligence, Peking University · JIUTIAN Research, China Mobile
Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Models
Sep 16, 2026Alizishaan Khatri, Chiquita Prabhu, Omkar NeogiSafety Filtering LLM Safety
Wrynx Inc. · Independent Researcher

Jul 14, 2026J/K move · Enter open · S save

Silent Alarm: A J-Space Protocol for Comparing Danger Recognition Across Models and Quantization Levels

Roman Prosvirnin, Victor Minchenkov, Alexey Soldatov, Vladimir Bashun

HSE University

LLM Safety Benchmarks Language Model Safety Evaluation

Open paper AskPDF arXiv

Jailbreak-robustness research typically evaluates safety through generated responses using an LLM-as-judge approach. Such evaluations, however, are sensitive to the benchmark's grading procedure and capture only observed behavior on a given set of attacks, without directly revealing the hidden fragility of the underlying safety mechanisms. This work proposes JADR (Jacobian Assessment of Danger Recognition), a protocol that measures a model's internal representation through Jacobian space (J-space, a recently proposed workspace of verbalizable concepts) before the first response token is generated. For every prompt and layer we record the top-k J-space tokens; these are grouped into six behavioral scenario axes and compared between a danger sample based on StrongREJECT and a safe control drawn from XSTest and OKTest. The method does not call on an external judge model: the computation runs entirely locally, on the activations of the model under evaluation, which lets us compare both different models against each other and modifications of a single model -- quantization and fine-tuning in particular -- on the same terms. The final comparison rests on the proposed SafetyAUC metric, complemented with bootstrap confidence intervals. The protocol is applied to six models (Qwen3-1.7B, Qwen3-4B, Qwen3-8B, Qwen3-Uncensored-4B, Qwen3-SafeRL-4B, Gemma 2 9B) across three weight-representation regimes -- BF16, INT8, and INT4 -- and checked against an independent behavioral evaluation with the StrongREJECT grader. The metric separates models with a strong versus a weak internal safety mechanism with statistical significance and captures substantively different effects across quantization regimes.

Abstract

LLM Safety

Explore similar work

Silent Alarm: A J-Space Protocol for Comparing Danger Recognition Across Models and Quantization Levels

DANGERLENS: Probing Dangerous Representations Hidden by Safety Alignment

Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Models