The Predictability Bias of Joint-Embedding Predictive Learning in Medical Imaging

Fabian Drexel; Bernhard Kainz; Jakob N. Kather; Daniel Rueckert; Lisa C. Adams; Keno K. Bressem; Cosmin I. Bercea

eess.IVOct 9, 2026

The Predictability Bias of Joint-Embedding Predictive Learning in Medical Imaging

Authors: Fabian Drexel, Bernhard Kainz, Jakob N. Kather, Daniel Rueckert, Lisa C. Adams, Keno K. Bressem, Cosmin I. Bercea

Organizations: Chair for AI in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany · Technical University of Munich, School of Medicine and Health, TUM University Hospital, Klinikum rechts der Isar, Department of Diagnostic and Interventional Radiology, Munich, Germany · Dept. AIBE, Friedrich-Alexander University Erlangen-Nürnberg, Erlangen, Germany · Department of Computing, Imperial College London, London, UK · Institute for Medical AI (IMAI), Heidelberg University Hospital and Medical Faculty of Heidelberg University, Heidelberg, Germany · Else Kroener Fresenius Center for Digital Health (EKFZ), Faculty of Medicine Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany · Munich Center for Machine Learning (MCML), Munich, Germany · Institute for Artificial Intelligence in Medicine (IKIM), University Hospital Essen (AöR), Essen, Germany · Institute of Interventional and Diagnostic Radiology and Neuroradiology, University Hospital Essen, Essen, Germany · National Center for Tumor Diseases West, Essen, Germany

Abstract

Joint-embedding predictive architectures (JEPAs) learn by predicting teacher representations of masked regions from visible context. Point prediction summarizes the conditional target distribution and cannot reproduce variation that the context does not resolve. We call this constraint predictability bias. In computed tomography (CT), recurrent anatomy is predictable from context, whereas the sporadic, localized pathology that determines clinical interpretation often is not. We introduce TAP-JEPA, a ViT-B/3D trained on 113,611 CT volumes that is competitive with state-of-the-art encoders (COLIPRI, Merlin, VoxelFM) in scan-level classification. We then audit the learned predictive pathway by replaying the exact masking, teacher, and predictor computations used during training. Prediction lowers the mean Skill (prevalence-normalized average precision) from 0.388 to 0.271 for pathology, but only from 0.921 to 0.873 for anatomy. Along the teacher direction, separating pathology from matched control regions in the same organ, prediction moves pathology 34.7% toward the controls and controls 13.6% toward pathology, with larger shifts farther from visible context. Replacing visible matched-control representations increases surrounding-target prediction loss more than fully replacing pathology representations across four pathology categories. Downstream, removing all final-layer pathology tokens reduces the decision margin by at most 0.45% in four of seven scan-level probes. Together, these results show that contextual predictability is not a sufficient proxy for clinical relevance in medical representation learning. Training objectives should therefore retain diagnostic features regardless of their predictability from surrounding anatomy.

Figures & tables

Figure 1: Predictability bias in medical imaging. Joint-embedding prediction preferentially preserves information that is constrained by visible context. Left, common JEPA masking schemes can leave most localized pathology outside the visible context. Right, hidden-target prediction in TAP-JEPA shifts pathology representations toward matched normal controls (values, see Appendix C ).

Domain	Encoder	Pretraining	AUROC $\uparrow$	AUPRC $\uparrow$
Thorax (RAD-ChestCT)	TAP-JEPA (ours)	Image SSL	0.7590 [0.7533,0.7649]	0.5268 [0.5178,0.5368]
	COLIPRI †	VL	0.7657 [0.7608,0.7718]	0.5255 [0.5172,0.5365]
	VoxelFM	Image SSL	0.7603 [0.7553,0.7658]	0.5192 [0.5117,0.5288]
Abdomen (Merlin)	TAP-JEPA (ours)	Image SSL	0.8714 [0.8622,0.8796]	0.6965 [0.6818,0.7158]
	Merlin	VL + EHR	0.8668 [0.8562,0.8750]	0.6825 [0.6675,0.7013]
	VoxelFM	Image SSL	0.8379 [0.8269,0.8472]	0.6422 [0.6276,0.6634]

Table 1: Frozen scan-level classification on held-out thoracic and abdominal benchmarks: macro AUROC/AUPRC with 95% bootstrap intervals; bold marks the highest point estimate. † Published official COLIPRI results; other rows are local evaluations. Comparator sources: Wald et al. (2025) , Moreno-Aguado et al. (2026) and Blankemeier et al. (2026) . SSL: self-supervised learning; VL: vision–language training; EHR: electronic health record supervision.

Figure 2: Prediction reduces pathology discrimination and pathology–control separation. A: Teacher and predictor Skill; percentages show normalized shortfall (Eq. 5 ). Geometry baselines are omitted from the plot. Anatomy uses class-macro Skill; pathology uses positive-class Skill on distinct token populations. B: Projections onto cross-fitted teacher pathology–control axes. Teacher control/pathology positions are 0/1; predictor positions are

D_{-}

and

1-D_{+}

(Eq. 6 ). Atelectasis/consolidation is highlighted in both panels. Pancreas lacks sufficient matched pairs for B. Further details and uncertainties are provided in Appendices F.7 – F.8 .

Panel of Table 2 — Table 2: Replacing visible control representations disrupts prediction of shared hidden regions more than replacing pathology representations in all six pathology categories, with four differences significant after multiplicity correction. Positive values indicate higher control replacement cost. Costs are adjusted for equal-count unrelated-region replacement and normalized by unadjusted control replacement cost or unperturbed target loss. Whiskers show pointwise 95% CIs; filled markers and ∗ denote joint-adjusted $p<0.05$ . † Below 100 pairs, descriptive. Full protocol in Appendix F.9 .

Figure 3: ATLAS token embeddings show anatomical and positional organization. A: Complete-view encoder tokens colored by anatomical structure. B: Each organ-specific UMAP is recolored by organ state and position. Disease-state coloring ranks pathology token fill within diseased organs, including zero-fill tokens; “healthy” denotes annotation-negative organs. Position uses organ-normalized coordinates mapped to RGB, with the right kidney mirrored to the left-side frame. Plotting and token-fill protocols: Appendix D.2 .

Panel of Table 3 — Table 3: Scan-level performance and dependence on local pathology tokens. Relative attention is the geometric mean of per-scan attention-per-token ratios for pathology versus surrounding organ/lobe tokens (log scale; $1\times$ = equal). Deletion effect is the median logit drop after removing all final-layer tokens overlapping same-category pathology annotations and renormalizing attention, expressed as a percentage of the positive–negative median-logit margin; 0/100% denotes no/full reference-margin loss. Whiskers show 95% group-bootstrap CIs. Exploratory $\rho_{\rm pos}$ is the volume-adjusted Spearman correlation between location typicality and deletion effect; ∗ denotes nominal $p<0.05$ . Details in Appendices F.2 – F.4 .

Dataset	Use in this study	Access and terms
NLST ( National Lung Screening Trial Research Team, 2011 ; Aberle et al., 2011 )	Pretraining	The Cancer Imaging Archive (TCIA), public collection ( National Lung Screening Trial Research Team, 2013 ; Clark et al., 2013 ) ; TCIA Data Usage Policy, CC BY 4.0
CT-RATE ( Hamamci et al., 2026 )	Pretraining; thoracic probe fitting, selection and source-domain evaluation	Hugging Face (gated); CC BY-NC-SA 4.0
Merlin ( Blankemeier et al., 2026 )	Pretraining; abdominal benchmark (SSL-held-out official test split)	Stanford AIMI; Stanford University School of Medicine Merlin Abdominal CT Dataset Research Use Agreement (personal, non-commercial research use)
AbdomenAtlas 3.0-mini (ATLAS) ( Bassi et al., 2025 )	Pretraining; abdominal regional analyses and MIL probes	Hugging Face; CC BY-NC-SA 4.0
ReXGroundingCT (ReX) ( Baharoon et al., 2026 )	Thoracic regional analyses and MIL probes	Hugging Face (gated); CC BY-NC-SA 4.0
RAD-ChestCT ( Draelos et al., 2021 ; Draelos et al., 2020 )	External thoracic evaluation only	Zenodo, restricted access approved by Duke University under CC BY-NC-ND 4.0; no further sharing of the data

Datasets used in this study, their role, and the access route and terms under which they are distributed by their publishers.

Appendix figures & tables37 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Specification
Input/patch	Three HU windows; voxel spacing XYZ $(1,1,1.5)$ mm; convolution kernel/stride ZYX $(16,24,24)$ , i.e. 24 mm isotropic tokens
Student/teacher	ViT-B: width 768, 12 layers, 12 heads; EMA teacher with saved output LayerNorm
Predictor/blocks	Width 384, 6 layers/heads; MLP ratio 4, GELU, LayerNorm, LayerScale ( Touvron et al., 2021 ) 0.1, no attention dropout/drop-path
Mask/loss	Approx. 16% context; remaining retained tokens targeted; feature-wise $L_{1}$ , weight $w_{i}=0.1+0.9\,o_{i}$ for body occupancy $o_{i}$ , normalized within volume (then: $\alpha_{i}$ ) then averaged equally across volumes
Optimizer	AdamW ( Loshchilov & Hutter, 2019 ) , betas $(0.9,0.95)$ , $\epsilon=10^{-8}$ , gradient clip 1; no weight decay on nonmatrix parameters or gradient accumulation
Batch	At most 48,000 tokens and 16 volumes/GPU; mean global batch 467.35 volumes on 32 GPUs; activation checkpointing

Appendix

Table 4: Production configuration and retained checkpoint. Compute is logged active training time, excluding queue/restart gaps, rather than complete scheduler billing.

Structure	CT-RATE		AbdomenAtlas
	AUROC	Skill	AUROC	Skill
Liver	0.993	0.964	0.978	0.915
Spleen	0.995	0.922	0.987	0.840
Left kidney	0.998	0.935	0.994	0.915
Right kidney	0.994	0.755	0.992	0.895
Pancreas	0.996	0.785	0.989	0.687

Appendix

Table 5: Structure-specific held-out anatomy-token results. Skill is prevalence-normalized AP (Section 4 ); entries are point estimates, with scan-bootstrap uncertainty described in the protocol.

Organ	$N$	$\pi$	Encoder AUROC	Encoder Skill	Acquisition Skill
Kidney	56,397	0.557	0.815 [0.789, 0.839]	0.607 [0.538, 0.662]	0.661 [0.599, 0.709]
Liver	79,845	0.414	0.697 [0.664, 0.730]	0.275 [0.217, 0.343]	0.477 [0.400, 0.554]
Pancreas	9,699	0.352	0.744 [0.706, 0.773]	0.413 [0.337, 0.477]	0.168 [0.089, 0.245]

Appendix

Table 6: ATLAS organ-state token probes. Brackets are 95% case-bootstrap CIs.

N

is evaluated tokens,

\pi

positive-token prevalence.

Dataset	Encoder	Pretraining	AUROC $\uparrow$	AUPRC $\uparrow$
CT-RATE	TAP-JEPA (ours)	Image SSL	0.8533 [0.8489, 0.8577]	0.6078 [0.5952, 0.6212]
	COLIPRI-CRM † ( Wald et al., 2025 )	VL	0.8612 [0.8552, 0.8685]	0.6128 [0.5959, 0.6310]
	VoxelFM ( Moreno-Aguado et al., 2026 )	Image SSL	0.8653 [0.8612, 0.8697]	0.6261 [0.6148, 0.6391]

Appendix

Table 7: Source-domain frozen scan-level classification on CT-RATE. Values are macro AUROC and macro AUPRC with 95% bootstrap confidence intervals. Bold marks the best point estimate and underline the runner-up. VL denotes vision–language pretraining; † denotes a published result obtained with a matched but non-identical readout and evaluation workflow.

Figure 4: Thoracic counterpart to Figure 3 . Left: CT-RATE anatomy tokens colored by 15 structures. Right: Per pathology family, ReX lung tokens versus report-normal CT-RATE lung tokens. The first coloring encodes organ state and ranks pathology fill within the family; its pale end includes diseased-lung tokens with no annotated pathology. The next two colorings show position in the whole-lung frame and lung side on the same organ-state embedding. The anatomy and organ-state panels use different token populations. Report-normal controls are annotation-negative. The anatomy legend is shared with Figure 3 ; abdominal and pelvic structures are only partially covered by chest CT.

		Pathology-token attention density	Token ablation (% margin)	Route / encoder Skill		Position $\rho_{\rm pos}$
Pathology	Skill			Acq.	Co-occur.
ATLAS
Liver lesion	0.376	2.1 $\times$ [1.7, 2.5]	0.45 [0.28, 0.65]	0.84	0.17	$+0.178^{*}$
Kidney lesion	0.584	0.65 $\times$ [0.57, 0.75]	0.18 [0.12, 0.28]	1.08	0.13	$+0.129^{*}$
Pancreatic lesion	0.386	0.99 $\times$ [0.79, 1.24]	1.69 [1.14, 2.51]	0.68	0.00	$+0.006$
ReX

Appendix

Table 8: Annotated-cohort MIL and alternative routes. Pathology-token attention density is the geometric mean of scan ratios to surrounding-organ/lobe attention; brackets are 95% group-bootstrap CIs. Ablation is median token deletion effect as a percentage of the fixed full-sample positive–negative median-logit margin; its intervals are 95% percentile group-bootstrap CIs from 1,000 replicates (Appendix F.2 ). Route/encoder Skill uses identical evaluation rows but separately cross-fitted routes. Position is the volume-adjusted scan-family association of largest-pathology typicality with all-pathology-token ablation. ∗ denotes matching nominal wild-cluster

p<0.05

(Appendix F.3 ).

Family	$n/G$	Necessity $\rho$	$p$	Attention $\rho$	$p$
ATLAS kidney	374/374	$+0.129$	0.01617	$+0.197$	0.00021
ATLAS liver	261/261	$+0.178$	0.00507	$+0.106$	0.09529
ATLAS pancreas	197/197	$+0.006$	0.93144	$-0.083$	0.23699
ReX scarring/fibrosis	230/213	$+0.032$	0.62385	$-0.355$	$<0.0001$
ReX atelectasis/consol.	301/269	$-0.036$	0.54999	$+0.021$	0.72843
ReX ground-glass	356/345	$+0.100$	0.06187	$+0.128$	0.01917

Appendix

Table 9: Typicality versus scan-family ablation cost or pathology-token attention, adjusting ranks for largest-pathology volume.

n/G

: rows / case or patient groups. Nominal conditional wild-cluster

p

values test the partial coefficients; no multiplicity adjustment.

Class	Organ-region	log 2	Skill by volume	Score $\sim$ vol.	Excess $\sim$ vol.
	attention	density	quartile Q1–Q4	$\rho$	$\rho$
Liver lesion	0.245	$+1.046$	0.091 / 0.070 / 0.184 / 0.432	$+0.302$	$+0.575$
Kidney lesion	0.077	$-0.630$	0.124 / 0.353 / 0.506 / 0.466	$+0.313$	$-0.075$
Pancreatic lesion	0.170	$-0.010$	0.081 / 0.271 / 0.263 / 0.237	$+0.190$	$+0.016$
Scarring/fibrosis	0.245	$-0.250$	0.044 / 0.074 / 0.116 / 0.122	$+0.050$	$+0.167$
Atelectasis/consol.	0.634	$+3.223$	0.136 / 0.256 / 0.428 / 0.596	$+0.490$	$+0.381$

Appendix

Table 10: Extended localization metrics. Organ-region attention is total attention on pathology plus its parent organ/lobes; density is the mean log 2 pathology-to-surrounding-tissue attention ratio. Score

\sim

vol. and excess

\sim

vol. correlate positive-scan scores and global pathology attention excess with union pathology volume. Skill quartiles share negatives; prevalence differs.

Figure 5: Production masks fully conceal most annotated pathologies in SSL-held-out cohorts. A: Fractions of crop-retained pathology–mask events that are fully visible, partially visible, or fully hidden and targeted; the three states sum to 100% before rounding. B: Median annotation-boundary-to-visible-token-box distance over fully hidden-targeted events. Whiskers are pointwise 95% confidence intervals from 2,000 case/study-group (ATLAS) or patient (ReX) bootstrap resamples, conditional on the frozen spatial draws. ReX uses the instance’s assigned dominant lung lobe for the same-organ distance. Full numbers, IQRs, crop retention, size associations and positive-target rates are in Appendix F.5 .

Dataset	Family	hidden-targeted (%)	Any context	Same organ/lobe	$\rho_{H}$ / $\rho_{d}$	Pos./1k
		Visible / partial /	Median distance to visible context (mm)
ATLAS	Kidney lesion	6.01 / 19.03 / 74.95	25.11 [11.94, 46.50]	29.89 [12.75, 78.54]	$-0.815$ / $-0.589$	2.12
ATLAS	Liver lesion	7.17 / 17.58 / 75.25	24.83 [11.50, 49.50]	24.50 [10.85, 47.25]	$-0.771$ / $-0.352$	1.19
ATLAS	Pancreatic lesion	3.22 / 26.90 / 69.88	21.45 [9.75, 39.32]	22.77 [8.37, 46.21]	$-0.805$ / $-0.572$	0.41
ReX	Scarring/fibrosis	2.74 / 30.38 / 66.88	21.62 [9.75, 43.50]	24.76 [10.50, 50.09]	$-0.808$ / $-0.435$	2.50
ReX	Atelectasis/consolidation	2.02 / 36.46 / 61.52	20.25 [8.71, 39.45]	21.46 [8.75, 41.38]	$-0.895$ / $-0.602$	5.89

Appendix

Table 11: Production-mask exposure. State probabilities condition on crop retention and sum to 100%. Distances are medians over fully hidden-targeted events with IQRs below, measured from the annotation boundary to the token box in millimetres after deployed preprocessing. Same-organ distances additionally condition on visible same-organ context being present; for ReX this is the assigned dominant lung lobe, and kidney support is the union of both kidneys.

\rho_{H}

and

\rho_{d}

are Spearman correlations of instance volume with concealment probability and with median context distance; every group-bootstrap interval excludes zero. These associations use all available annotation protocols, including the small ReX exhaustive stratum, as in the original analysis. “Pos./1k” is the family’s cohort-level positive-target-token contribution and is an annotated-cohort rate, not a pretraining-mixture prevalence. ReX rows use the 909-scan partial representative-annotation protocol with CT-RATE lobe anatomy. Instances retained after deployed preprocessing and cropping, out of all eligible: kidney 1,255/1,309, liver 971/1,025, pancreas 281/282, scarring/fibrosis 525/527, atelectasis/consolidation 816/816, ground-glass 1,030/1,030, nodules/masses 947/948; crop exclusion was 4.13%, 5.27%, and 0.35% for kidney, liver, and pancreas, respectively, and 0–0.38% across ReX.

Dataset	Family	Pairs	Groups	AUROC
ATLAS	Kidney	671	280	0.692 [0.665,0.720]
ATLAS	Liver	808	261	0.713 [0.684,0.741]
ReX	Scarring/fibrosis	400	185	0.939 [0.923,0.955]
ReX	Atelectasis/consolidation	457	189	0.914 [0.890,0.938]
ReX	Ground-glass opacity	723	264	0.883 [0.859,0.904]
ReX	Nodules/masses	942	379	0.644 [0.622,0.665]

Appendix

Table 12: Original cross-fitted normalized EMA-teacher gate. ReX uses positive–unlabelled evaluation. Controls are unannotated, not verified healthy. Brackets are 95% intervals from 2,000 group bootstraps.

Figure 6: Detailed hidden-target recovery and matched-region displacement. A: Separate probes read geometry (G), prediction (P) and teacher (T). Bars show absolute Skill for pooled anatomy/pathology tasks in ATLAS and ReX, then seven pathology categories. Recovery $R_{\mathrm{rec}}=1-Q_{\mathrm{short}}$ is the fraction of the teacher–geometry Skill gap recovered by prediction; Eq. 5 defines the complementary main-text shortfall. The nodule/mass ratio is descriptive because the teacher–geometry gap is below 0.10. B: Both the pathology region and its matched annotation-negative control are hidden. Pooled region features are projected onto a cross-fitted teacher pathology–control axis. Retained contrast $C_{\mathrm{ret}}=1-A_{\mathrm{ctr}}$ and signed displacements $D_{+}$ and $D_{-}$ sum to one; Eq. 6 defines the main-text attenuation $A_{\mathrm{ctr}}=D_{+}+D_{-}$ . Whiskers show 95% intervals for retained contrast. Pancreatic lesions lack sufficient eligible matched pairs for B. Schematics illustrate the protocols; full estimates and uncertainty are reported in this appendix and Appendix F.8 .

Dataset	Scope	Source scans	Eval. scans	Groups	Rows	$\pi$ (%)
ATLAS	Pooled	768	756	756	847,565	14.12
ATLAS	Kidney lesions	440	430	430	318,093	21.72
ATLAS	Liver lesions	297	292	292	485,387	7.82
ATLAS	Pancreatic lesions	224	221	221	58,141	22.29
ATLAS	Kidney/liver union	606	595	595	791,784	13.52
ReX	Pooled	887	887	813	2,122,126	8.95

Appendix

Table 13: Hidden-target recovery pathology-task cohorts and positive prevalence

\pi

(%). Source counts are in-scope annotated candidates; evaluated scans have at least one eligible hidden-target pathology or control row. Groups are ATLAS studies and ReX patients. Rows are token–mask observations, not independent scans; family cohorts overlap. The pooled populations contain 119,656/189,857 positive and 727,909/1,932,269 negative rows in ATLAS/ReX.

ATLAS structure	$\pi$ (%)	ReX structure	$\pi$ (%)
Liver	38.69	Left upper lobe	15.70
Spleen	3.32	Left lower lobe	17.26
Left kidney	13.53	Right upper lobe	12.19
Right kidney	12.38	Right middle lobe	5.42
Pancreas	4.26	Right lower lobe	15.91
Stomach	4.88	Trachea	1.23

Appendix

Table 14: Class prevalences (%) in the pooled anatomy tasks: 1,033,142 ATLAS and 2,316,592 ReX token–mask rows, on the same 756/887 eligible scans as their pathology tasks. Structures are listed in label order; horizontal alignment across datasets has no anatomical meaning.

Dataset	Task	Geometry $\mathcal{B}$	Prediction $\widehat{Z}$	Teacher $Z$	$R_{\mathrm{rec}}$
ATLAS	Anatomy	0.294 [0.282, 0.308]	0.820 [0.812, 0.828]	0.886 [0.879, 0.893]	0.888 [0.881, 0.894]
ATLAS	Pathology status	0.064 [0.042, 0.093]	0.298 [0.246, 0.349]	0.434 [0.386, 0.482]	0.633 [0.561, 0.686]
ReX	Anatomy	0.456 [0.449, 0.464]	0.925 [0.923, 0.928]	0.956 [0.954, 0.958]	0.938 [0.934, 0.941]
ReX	Pathology status	0.034 [0.026, 0.043]	0.243 [0.214, 0.275]	0.341 [0.311, 0.374]	0.682 [0.639, 0.724]

Appendix

Table 15: Expanded hidden-target recovery pooled results with the spline geometry floor. Values are Skill and recovery, with pointwise 95% group-bootstrap intervals. Anatomy uses class-macro Skill; pathology status uses positive-class Skill. The four rows correspond to the pooled tasks in Figure 6 A; only ATLAS anatomy is shown in the main Figure 2 A.

Dataset	Pathology	Geometry	Prediction	Teacher	$G$	$R_{\mathrm{rec}}$
ATLAS	Kidney lesions	0.086 [0.046, 0.132]	0.427 [0.343, 0.495]	0.580 [0.508, 0.639]	0.493	0.690 [0.613, 0.748]
ATLAS	Liver lesions	0.012 [0.003, 0.023]	0.153 [0.092, 0.228]	0.325 [0.241, 0.411]	0.313	0.452 [0.323, 0.560]
ATLAS	Pancreatic lesions	0.014 [-0.010, 0.041]	0.181 [0.127, 0.240]	0.229 [0.175, 0.289]	0.215	0.779 [0.610, 0.932]
ReX	Scarring/fibrosis	0.038 [0.022, 0.058]	0.186 [0.150, 0.229]	0.247 [0.203, 0.294]	0.209	0.709 [0.626, 0.794]
ReX	Atelectasis/consolidation	0.062 [0.042, 0.087]	0.293 [0.245, 0.336]	0.428 [0.380, 0.473]	0.366	0.630 [0.561, 0.696]
ReX	Ground-glass opacity	0.033 [0.022, 0.046]	0.203 [0.174, 0.234]	0.319 [0.286, 0.353]	0.286	0.595 [0.536, 0.648]

Appendix

Table 16: Pathology-specific hidden-target recovery positive-class Skills, teacher–geometry gap

G

, and recovery, with pointwise 95% intervals. Each family has its own probe fits.

\dagger

The nodule/mass gap is below 0.10, so its ratio is descriptive. Family cohorts overlap and these scores are not averaged to obtain the pooled result.

Dataset	Task	Geometry Skill	$R_{\mathrm{rec}}$
ATLAS	Anatomy	0.309 [0.296, 0.322]	0.885 [0.878, 0.892]
ATLAS	Pathology status	0.063 [0.039, 0.090]	0.635 [0.570, 0.689]
ReX	Anatomy	0.493 [0.486, 0.501]	0.933 [0.929, 0.937]
ReX	Pathology status	0.029 [0.022, 0.037]	0.687 [0.642, 0.727]

Appendix

Table 17: Richer geometry-floor sensitivity on identical hidden-target recovery rows and folds. Prediction and teacher scores are unchanged; the alternative floor combines standardized geometry, 256 random Fourier features and mask-family indicators.

Dataset	Geometry floor	$R_{\mathrm{rec},\mathrm{anatomy}}-R_{\mathrm{rec},\mathrm{pathology}}$
ATLAS	Spline	0.255 [0.202, 0.325]
ATLAS	Fourier	0.251 [0.196, 0.316]
ReX	Spline	0.256 [0.214, 0.299]
ReX	Fourier	0.246 [0.205, 0.290]

Appendix

Table 18: Paired anatomy-minus-pathology recovery differences. Shared study/patient bootstrap draws pair the uncertainty, while anatomy and pathology status retain their individual token populations and score aggregation.

Family	$N/G$	$C_{\mathrm{ret}}$	$D_{+}$	$D_{-}$	$D_{+}-D_{-}$
ATLAS kidney	671/280	0.564 [0.523, 0.602]	0.154 [0.119, 0.191]	0.282 [0.247, 0.321]	-0.128 [-0.192, -0.069]
ATLAS liver	808/261	0.276 [0.233, 0.319]	0.548 [0.497, 0.595]	0.177 [0.147, 0.208]	0.371 [0.298, 0.436]
ReX scarring/fibrosis	400/185	0.821 [0.795, 0.846]	0.137 [0.112, 0.166]	0.042 [0.023, 0.060]	0.095 [0.060, 0.137]
ReX atelectasis/consol.	457/189	0.455 [0.416, 0.493]	0.482 [0.439, 0.523]	0.063 [0.041, 0.086]	0.419 [0.364, 0.471]
ReX ground-glass	723/264	0.426 [0.400, 0.451]	0.499 [0.468, 0.529]	0.076 [0.054, 0.095]	0.423 [0.378, 0.467]
ReX nodules/masses	942/379	0.558 [0.520, 0.594]	0.265 [0.230, 0.299]	0.177 [0.149, 0.206]	0.088 [0.036, 0.138]

Appendix

Table 19: Primary matched-region displacement estimates and pointwise 95% group-bootstrap intervals.

N/G

: region pairs / patient or study groups. Signed displacement is measured in teacher-contrast units.

Family	Random mean	Central 95% of axes	Pathology minus mean	Rank (%)
ATLAS kidney	0.480 [0.467, 0.493]	[0.406, 0.558]	0.083 [0.046, 0.118]	98.2
ATLAS liver	0.443 [0.426, 0.459]	[0.370, 0.520]	-0.167 [-0.202, -0.131]	0.0
ReX scarring/fibrosis	0.643 [0.624, 0.660]	[0.551, 0.738]	0.178 [0.158, 0.198]	100.0
ReX atelectasis/consol.	0.525 [0.501, 0.549]	[0.413, 0.634]	-0.070 [-0.097, -0.041]	9.7
ReX ground-glass	0.564 [0.550, 0.579]	[0.461, 0.663]	-0.139 [-0.159, -0.119]	0.5
ReX nodules/masses	0.572 [0.560, 0.585]	[0.506, 0.646]	-0.014 [-0.047, 0.019]	36.1

Appendix

Table 20: Generic calibration using 4,096 random-axis sets. Bracketed intervals under means and differences are group-bootstrap CIs; the central axis range describes variation between directions. Rank is the percentage of sampled coefficients below the pathology coefficient, not a

p

-value.

Family	$g$	$C_{\mathrm{ret}}-g$	Adj. $p$	Energy ratio
ATLAS kidney	0.483 [0.469, 0.496]	0.081 [0.044, 0.116]	0.012	1.16
ATLAS liver	0.445 [0.427, 0.461]	-0.169 [-0.204, -0.133]	0.012	2.74
ReX scarring/fibrosis	0.649 [0.630, 0.667]	0.173 [0.153, 0.191]	0.012	19.89
ReX atelectasis/consol.	0.526 [0.500, 0.550]	-0.071 [-0.097, -0.042]	0.012	13.42
ReX ground-glass	0.565 [0.550, 0.581]	-0.140 [-0.160, -0.121]	0.012	8.54
ReX nodules/masses	0.575 [0.562, 0.588]	-0.016 [-0.050, 0.016]	0.378	0.97

Appendix

Table 21: Exact isotropic moment reference

g

, paired differences and six-family Holm-adjusted approximate bootstrap

p

-values. The repeated 0.012 values reflect finite bootstrap resolution. Energy ratio is pathology-axis teacher-contrast energy divided by mean random-axis energy.

Family	$N/G$	Normalized pathology excess
ATLAS kidney	446/222	0.953 [0.763, 1.142]
ATLAS liver	674/226	1.369 [1.134, 1.631]
ReX scarring/fibrosis	254/136	3.970 [3.516, 4.477]
ReX atelectasis/consol.	280/138	2.814 [2.445, 3.283]
ReX ground-glass	498/217	2.758 [2.456, 3.080]
ReX nodules/masses	881/367	0.942 [0.836, 1.055]

Appendix

Table 22: Teacher sensitivity with token-disjoint original/alternative-1 controls. Pathology excess is in RMS control-spread units, with fixed-gate 95% case/patient-bootstrap intervals.

N/G

: pairs / groups.

Family	Pairs	Original	Alt. 1	Consensus	Cosine	Control generator
ATLAS kidney	498	0.665	0.661	0.688	0.515	0.525 [0.491, 0.560]
ATLAS liver	756	0.710	0.742	0.773	0.590	0.508 [0.483, 0.535]
ReX scarring/fibrosis	361	0.939	0.960	0.969	0.681	0.661 [0.631, 0.693]
ReX atelectasis/consol.	388	0.907	0.942	0.957	0.677	0.566 [0.532, 0.601]
ReX ground-glass	636	0.879	0.915	0.933	0.692	0.584 [0.555, 0.613]
ReX nodules/masses	930	0.644	0.745	0.802	0.511	0.605 [0.579, 0.629]

Appendix

Table 23: Teacher-axis sensitivity on the subset with the required alternative control. Columns report held-out pathology/control AUROC, original–alternative direction cosine (median across folds), and the AUROC for distinguishing alternative-1 from original control tissue.

Family	$N/G$	$C_{\mathrm{ret}}$	$D_{+}$	$D_{-}$	$D_{+}-D_{-}$
ATLAS kidney	298/172	$[0.514,0.621]$	$[0.100,0.175]$	$[0.270,0.320]$	$[-0.203,-0.136]$
ATLAS liver	683/231	$[0.285,0.391]$	$[0.431,0.517]$	$[0.164,0.211]$	$[0.253,0.319]$
ReX scarring/fibrosis	273/141	$[0.832,0.875]$	$[0.095,0.127]$	$[0.031,0.048]$	$[0.058,0.090]$
ReX atelectasis/consol.	289/131	$[0.435,0.502]$	$[0.421,0.505]$	$[0.058,0.101]$	$[0.324,0.445]$
ReX ground-glass	513/208	$[0.391,0.500]$	$[0.442,0.533]$	$[0.055,0.093]$	$[0.371,0.458]$
ReX nodules/masses	911/374	$[0.546,0.808]$	$[0.105,0.269]$	$[0.083,0.185]$	$[-0.006,0.083]$

Appendix

Table 24: Common-pair, common-mask sensitivity: ranges across four controls crossed with three fixed teacher axes. Entries are min–max point estimates, not confidence intervals.

N/G

is identical across all 12 cells within a family.

Family	$[0,24)$	$[24,48)$	$[48,96)$	$[96,\infty)$
ATLAS kidney	0.674 [0.633, 0.712]; $n=671$	0.533 [0.490, 0.574]; $n=671$	0.440 [0.392, 0.491]; $n=669$	0.407 [0.350, 0.459]; $n=486$
ATLAS liver	0.326 [0.283, 0.371]; $n=808$	0.263 [0.222, 0.309]; $n=805$	0.222 [0.176, 0.268]; $n=803$	0.166 [0.109, 0.232]; $n=614$
ReX scarring/fibrosis	0.849 [0.826, 0.871]; $n=400$	0.810 [0.782, 0.838]; $n=400$	0.786 [0.757, 0.814]; $n=396$	0.802 [0.761, 0.843]; $n=231$
ReX atelectasis/consol.	0.484 [0.444, 0.525]; $n=457$	0.434 [0.394, 0.478]; $n=457$	0.421 [0.383, 0.463]; $n=456$	0.420 [0.369, 0.470]; $n=194$
ReX ground-glass	0.448 [0.423, 0.475]; $n=723$	0.411 [0.385, 0.437]; $n=723$	0.395 [0.371, 0.421]; $n=719$	0.387 [0.352, 0.425]; $n=324$
ReX nodules/masses	0.606 [0.567, 0.644]; $n=942$	0.547 [0.505, 0.587]; $n=942$	0.497 [0.452, 0.539]; $n=942$	0.471 [0.424, 0.518]; $n=633$

Appendix

Table 25: Distance-stratified coefficients with 95% group-bootstrap intervals and pair counts. Distance is the region-boundary-to-nearest-visible-token distance in millimetres; bins are left-closed and right-open. Roles are filtered separately before pair-level averaging.

Family	Multi-block target	Multi-scale blocks	Single-crop context
ATLAS kidney	0.589 [0.548, 0.627]	0.586 [0.544, 0.624]	0.510 [0.464, 0.551]
ATLAS liver	0.294 [0.251, 0.340]	0.290 [0.247, 0.338]	0.245 [0.198, 0.296]
ReX scarring/fibrosis	0.833 [0.808, 0.858]	0.834 [0.808, 0.857]	0.796 [0.769, 0.825]
ReX atelectasis/consol.	0.463 [0.423, 0.503]	0.458 [0.418, 0.498]	0.442 [0.403, 0.481]
ReX ground-glass	0.431 [0.405, 0.456]	0.427 [0.400, 0.453]	0.417 [0.390, 0.442]
ReX nodules/masses	0.574 [0.536, 0.611]	0.570 [0.531, 0.608]	0.526 [0.486, 0.566]

Appendix

Table 26: Coefficients and 95% group-bootstrap intervals within each production mask family. These are conditional strata, not interventions on the training mask distribution.

Family	Effective groups	Max. weight (%)	Leave-one-out $C_{\mathrm{ret}}$	Mean MCSE	Max. tail MCSE
ATLAS kidney	105.5	2.71	[0.559, 0.570]	0.0008	0.0021
ATLAS liver	79.6	3.05	[0.268, 0.283]	0.0005	0.0020
ReX scarring/fibrosis	92.9	3.88	[0.818, 0.824]	0.0007	0.0028
ReX atelectasis/consol.	88.0	3.88	[0.447, 0.462]	0.0008	0.0030
ReX ground-glass	105.6	2.82	[0.423, 0.429]	0.0007	0.0022
ReX nodules/masses	123.2	4.03	[0.552, 0.563]	0.0004	0.0017

Appendix

Table 27: Primary stability diagnostics. Effective groups and largest group weight use squared teacher-gap weights. Leave-one-out ranges delete one patient/study group. Monte Carlo standard errors (MCSE) use eight independent blocks of 512 random-axis sets.

Family	Source pairs	O pairs	O groups	O events	S pairs	S groups	S events
ATLAS kidney	143	62	50	573	60	48	555
ATLAS liver	417	405	206	4947	402	206	4900
ReX scarring/fibrosis	281	243	141	2800	242	141	2784
ReX atelectasis/ consolidation	327	282	149	3377	280	149	3353
ReX ground glass	505	447	219	5187	444	218	5146
ReX nodules/masses	321	290	205	2936	288	204	2918

Appendix

Table 28: Visible-context intervention source and retained support. O: own neighbourhoods; S: shared targets. Groups are ATLAS released studies/cases or ReX patients. Each view requires eight events per pair. Kidney is below the 100-pair threshold.

Family	Targets	$n$	$\Delta\times 10^{3}$	$R_{\mathrm{ctrl}}$ (%)	$100\Delta/\langle B\rangle$	Adj. $p$
ATLAS kidney	Own	62	$-1.209$ $[-2.836,0.440]$	$9.5$ $[-3.6,21.3]$	$-0.335$ $[-0.792,0.121]$	—
ATLAS kidney	Shared	60	$-1.525$ $[-3.270,0.252]$	$11.6$ $[-2.0,24.0]$	$-0.423$ $[-0.913,0.068]$	—
ATLAS liver	Own	405	$-0.977$ $[-1.726,-0.302]$	$6.7$ $[2.2,11.2]$	$-0.273$ $[-0.481,-0.085]$	$0.0114$
ATLAS liver	Shared	402	$-2.124$ $[-3.052,-1.273]$	$12.8$ $[8.1,17.3]$	$-0.599$ $[-0.860,-0.362]$	$<0.001$
ReX scarring/fibrosis	Own	243	$-1.176$ $[-2.410,0.024]$	$5.3$ $[-0.1,10.4]$	$-0.352$ $[-0.719,0.007]$	$0.0720$
ReX scarring/fibrosis	Shared	242	$-8.064$ $[-9.900,-6.346]$	$27.3$ $[22.4,32.0]$	$-2.433$ $[-2.989,-1.914]$	$<0.001$

Appendix

Table 29: Both target definitions with the joint reporting correction spanning four ATLAS and eight ReX contrasts. Intervals are pointwise; kidney is descriptive. Baseline percentage uses the signed adjusted contrast and therefore has the opposite sign to

R_{\mathrm{ctrl}}

Family	Targets	$\langle B\rangle$	$\langle A_{+}\rangle\!\times\!10^{3}$	$\langle A_{-}\rangle\!\times\!10^{3}$	$\Delta_{\mathrm{raw}}\!\times\!10^{3}$	$R_{\mathrm{raw}}$ (%)
ATLAS kidney	Own	$0.361$	$11.174$	$12.740$	$-1.566$ $[-3.163,0.032]$	$12.3$ $[-0.3,23.7]$
ATLAS kidney	Shared	$0.360$	$11.363$	$13.193$	$-1.830$ $[-3.547,-0.154]$	$13.9$ $[1.2,25.8]$
ATLAS liver	Own	$0.358$	$13.520$	$14.575$	$-1.056$ $[-1.836,-0.367]$	$7.2$ $[2.7,12.0]$
ATLAS liver	Shared	$0.354$	$14.631$	$16.605$	$-1.974$ $[-2.900,-1.136]$	$11.9$ $[7.2,16.5]$
ReX scarring/fibrosis	Own	$0.334$	$21.717$	$22.362$	$-0.645$ $[-1.834,0.534]$	$2.9$ $[-2.5,8.0]$
ReX scarring/fibrosis	Shared	$0.331$	$22.176$	$29.549$	$-7.373$ $[-9.157,-5.703]$	$25.0$ $[20.1,29.6]$

Appendix

Table 30: Absolute baseline loss, role-specific mean replacement costs, raw contrasts and raw relative reduction.

A_{+}

and

A_{-}

are unadjusted costs. Raw contrast/reduction intervals are pointwise and are not the tests of reference-adjusted

\Delta

Family	Pairs O/S	AUROC O	AUROC S	AP S	Skill S	Gate S
ATLAS kidney	62/60	$0.681$ $[0.599,0.767]$	$0.679$ $[0.588,0.771]$	$0.677$	$0.353$	Support fail
ATLAS liver	405/402	$0.801$ $[0.772,0.835]$	$0.801$ $[0.770,0.834]$	$0.819$	$0.638$	Pass
ReX scarring/fibrosis	243/242	$0.950$ $[0.931,0.968]$	$0.950$ $[0.930,0.967]$	$0.946$	$0.893$	Pass
ReX atelectasis/ consolidation	282/280	$0.930$ $[0.902,0.955]$	$0.931$ $[0.903,0.956]$	$0.929$	$0.858$	Pass
ReX ground glass	447/444	$0.909$ $[0.885,0.929]$	$0.908$ $[0.885,0.928]$	$0.911$	$0.823$	Pass
ReX nodules/masses	290/288	$0.684$ $[0.649,0.720]$	$0.681$ $[0.645,0.717]$	$0.694$	$0.388$	Pass

Appendix

Table 31: Actual pre-replacement masked-student diagnostic on own (O) and shared (S) event support, using unchanged held-out directions. Each pair supplies one pathology and one control (prevalence 0.5). Shared positive-class AP and Skill are point estimates; AUROC intervals use 2,000 group resamples.

Family	Pairs	Events	Own targets	Shared targets
ATLAS kidney	60	555	$-1.147$ $[-2.764,0.487]$	$-1.525$ $[-3.270,0.252]$
ATLAS liver	402	4900	$-0.994$ $[-1.726,-0.327]$	$-2.124$ $[-3.052,-1.273]$
ReX scarring/fibrosis	242	2784	$-1.131$ $[-2.369,0.069]$	$-8.064$ $[-9.900,-6.346]$
ReX atelectasis/ consolidation	280	3353	$-1.742$ $[-2.995,-0.502]$	$-5.535$ $[-7.192,-3.890]$
ReX ground glass	444	5146	$-0.457$ $[-1.337,0.465]$	$-3.847$ $[-4.935,-2.713]$
ReX nodules/masses	288	2918	$-0.182$ $[-0.949,0.588]$	$-0.975$ $[-1.937,-0.042]$

Appendix

Table 32: Own and shared target readouts on identical eligible pair/mask events. Values are adjusted contrasts in

10^{-3}

loss units with pointwise intervals. The events and interventions are shared; evaluated targets differ.

Family	Donor own	Donor shared	Equal norm own	Equal norm shared
ATLAS kidney	$-0.351$ $[-1.473,0.807]$	$-0.523$ $[-1.773,0.742]$	$-0.614$ $[-1.921,0.732]$	$-0.791$ $[-2.167,0.613]$
ATLAS liver	$-0.944$ $[-1.441,-0.487]$	$-1.438$ $[-1.994,-0.927]$	$-0.756$ $[-1.361,-0.196]$	$-1.404$ $[-2.094,-0.736]$
ReX scarring/fibrosis	$-1.017$ $[-1.690,-0.385]$	$-4.316$ $[-5.249,-3.427]$	$-0.979$ $[-2.011,0.021]$	$-6.257$ $[-7.839,-4.752]$
ReX atelectasis/ consolidation	$-1.293$ $[-1.931,-0.664]$	$-3.087$ $[-3.934,-2.288]$	$-0.875$ $[-1.955,0.141]$	$-3.969$ $[-5.281,-2.642]$
ReX ground glass	$-0.746$ $[-1.178,-0.303]$	$-2.178$ $[-2.737,-1.626]$	$-0.155$ $[-0.960,0.662]$	$-2.758$ $[-3.709,-1.763]$
ReX nodules/masses	$-0.253$ $[-0.632,0.130]$	$-0.561$ $[-1.035,-0.098]$	$0.112$ $[-0.582,0.837]$	$-0.522$ $[-1.377,0.311]$

Appendix

Table 33: Replacement sensitivities in

10^{-3}

loss units, with pointwise intervals. Donor: reference-adjusted contrast using sampled donor activations. Equal norm: raw contrast after scaling to a common Frobenius norm. Both retain the respective own/shared support in Table 28 .

Family	Region RMS $\times 10^{3}$	Region Frobenius	Reference RMS $\times 10^{3}$
ATLAS kidney	$-1.250$ $[-10.811,9.199]$	$-0.595$ $[-1.577,0.486]$	$-0.454$ $[-1.514,0.615]$
ATLAS liver	$12.203$ $[8.566,15.813]$	$0.651$ $[0.160,1.157]$	$0.001$ $[-0.320,0.320]$
ReX scarring/fibrosis	$24.470$ $[19.295,29.904]$	$0.692$ $[-0.099,1.493]$	$-0.150$ $[-0.612,0.289]$
ReX atelectasis/ consolidation	$16.475$ $[12.499,20.517]$	$0.348$ $[-0.539,1.204]$	$0.039$ $[-0.323,0.432]$
ReX ground glass	$9.765$ $[6.678,12.899]$	$0.140$ $[-0.445,0.713]$	$-0.060$ $[-0.353,0.224]$
ReX nodules/masses	$3.867$ $[0.280,7.464]$	$0.002$ $[-0.520,0.505]$	$0.118$ $[-0.283,0.524]$

Appendix

Table 34: Pathology-minus-control perturbation differences on the own-neighbourhood cohort. RMS columns are scaled by

10^{3}

; Frobenius differences are unscaled. These are activation-change norms, not prediction loss. Magnitudes are identical between BF16 and FP32 replay.

Family	Targets	Pairs	$\alpha=0.25$	$\alpha=0.5$	$\alpha=1$
ATLAS kidney	Own	62	$-0.123$ $[-0.244,-0.010]$	$-0.272$ $[-0.585,0.030]$	$-1.566$ $[-3.157,0.048]$
ATLAS kidney	Shared	60	$-0.094$ $[-0.211,0.016]$	$-0.257$ $[-0.571,0.063]$	$-1.830$ $[-3.547,-0.154]$
ATLAS liver	Own	405	$0.108$ $[0.062,0.156]$	$0.314$ $[0.178,0.454]$	$-1.056$ $[-1.826,-0.342]$
ATLAS liver	Shared	402	$0.103$ $[0.058,0.149]$	$0.216$ $[0.074,0.355]$	$-1.974$ $[-2.900,-1.136]$
ReX scarring/fibrosis	Own	243	$0.139$ $[0.081,0.199]$	$0.862$ $[0.628,1.107]$	$-0.645$ $[-1.807,0.519]$
ReX scarring/fibrosis	Shared	242	$0.057$ $[-0.009,0.125]$	$0.155$ $[-0.111,0.420]$	$-7.373$ $[-9.157,-5.703]$

Appendix

Table 35: Raw pathology-minus-control contrasts under partial and full mean replacement, in

10^{-3}

loss units with pointwise intervals. Each row retains the same events across doses. No reference adjustment is applied to the partial-dose contrasts.

Family	Targets	Control 0	Control 1	Control 2	Control 3
ATLAS kidney	Own	$-1.209$ ( $n=62$ ) $[-2.836,0.440]$	$-2.327$ ( $n=18$ ) $[-4.764,0.039]$	$-4.534$ ( $n=13$ ) $[-7.558,-1.773]$	$-3.506$ ( $n=9$ ) $[-9.077,1.871]$
ATLAS kidney	Shared	$-1.525$ ( $n=60$ ) $[-3.270,0.252]$	$-2.596$ ( $n=17$ ) $[-5.109,-0.250]$	$-4.889$ ( $n=12$ ) $[-8.193,-1.789]$	$-2.605$ ( $n=7$ ) $[-5.459,0.547]$
ATLAS liver	Own	$-0.977$ ( $n=405$ ) $[-1.726,-0.302]$	$-1.678$ ( $n=178$ ) $[-3.213,-0.343]$	$-1.721$ ( $n=165$ ) $[-3.208,-0.469]$	$-2.323$ ( $n=146$ ) $[-3.495,-1.276]$
ATLAS liver	Shared	$-2.124$ ( $n=402$ ) $[-3.052,-1.273]$	$-1.836$ ( $n=173$ ) $[-3.335,-0.503]$	$-2.294$ ( $n=155$ ) $[-3.564,-1.132]$	$-3.875$ ( $n=143$ ) $[-5.463,-2.458]$
ReX scarring/fibrosis	Own	$-1.176$ ( $n=243$ ) $[-2.410,0.024]$	$-3.339$ ( $n=116$ ) $[-5.481,-1.311]$	$-2.882$ ( $n=95$ ) $[-5.084,-0.756]$	$-2.955$ ( $n=79$ ) $[-5.588,-0.454]$
ReX scarring/fibrosis	Shared	$-8.064$ ( $n=242$ ) $[-9.900,-6.346]$	$-8.734$ ( $n=115$ ) $[-11.271,-6.180]$	$-9.456$ ( $n=94$ ) $[-11.926,-7.061]$	$-9.939$ ( $n=76$ ) $[-12.515,-7.454]$

Appendix

Table 36: Control-specific adjusted contrasts (

\times 10^{3}

), retained pairs and pointwise intervals. Control 0 repeats the primary reporting intervals from Table 29 . Each cell uses its own eligible cohort, so columns do not isolate control choice. Entries with fewer than 100 pairs are descriptive.

Family	Token disjoint	Regression intercept
ATLAS kidney	$-2.419$ ( $n=38$ ) $[-4.754,0.036]$	$-0.042$ $[-1.793,1.724]$
ATLAS liver	$-1.209$ ( $n=300$ ) $[-2.143,-0.340]$	$-0.262$ $[-0.833,0.308]$
ReX scarring/fibrosis	$-1.326$ ( $n=201$ ) $[-2.741,0.039]$	$0.533$ $[-1.391,2.786]$
ReX atelectasis/ consolidation	$-1.297$ ( $n=224$ ) $[-2.778,0.124]$	$0.637$ $[-1.240,2.360]$
ReX ground glass	$-0.586$ ( $n=328$ ) $[-1.712,0.519]$	$1.210$ $[0.207,2.288]$
ReX nodules/masses	$-0.080$ ( $n=192$ ) $[-1.156,0.983]$	$0.758$ $[0.064,1.500]$

Appendix

Table 37: Own-neighbourhood sensitivities: token-disjoint events and the descriptive covariate-adjusted intercept. Adjusted contrasts are scaled by

10^{3}

with pointwise intervals. The regression retains original own-neighbourhood support; its intercept is a model-based contrast at equal count, distance and perturbation-magnitude covariates, without multiplicity correction. Kidney token-disjoint support is below 100 pairs.

Explore similar work

Jul 28, 2026

Rad-JEPA 3D: Radiology Joint-Embedding Predictive Model for 3D Computed Tomography

Self-supervised pretraining is central to 3D medical image analysis, where unlabeled CT volumes are abundant but expert annotations are scarce. Yet existing volumetric encoders often fail to preserve the coarse spatial and geometric structure that downstream reasoning depends on, limiting their performance on organ disentanglement, abnormality detection, and spatial understanding when paired with language models. We introduce Rad-JEPA 3D, a joint-embedding predictive framework that learns volumetric CT representations by predicting the latent features of a complete scan from a masked view. At its core is a hybrid H-Mamba encoder that fuses a Mamba state-space branch, which models inter-slice continuity through sequential scanning, with a grouped-query attention branch, which captures cross-plane spatial context, combined through a lightweight per-token router. To improve the quality of intermediate representations, we further propose Hidden States Orthogonal Regularization (HSOR), which aligns student-teacher hidden states and reduces feature redundancy throughout the encoder. This layer-wise regularization produces more consistent and discriminative volumetric representations, leading to improved performance on organ recognition and spatial reasoning tasks. Pretrained on approximately 120,000 CT scans, Rad-JEPA 3D attains state-of-the-art results despite its compact size: with only 4.0B total parameters, it achieves competitive results with state-of-the-art on closed-ended VQA and the best average spatial-reasoning score on the Spatial-Med benchmark. Ablation studies confirm that the hybrid block and HSOR contribute complementary gains, and that the induced spatial structure can substitute for raw language-model scale on volumetric reasoning tasks.

Quoc-Huy Trinh, Minh-Van Nguyen, Ulas Bagci

Northwestern University · Technical University of Denmark

Medical Imaging Foundation Models 3D Medical Imaging Joint-Embedding Predictive Architecture 3D Representation Learning Medical Image Representation Learning

Ask PDF

Aug 28, 2026

Do Medical Vision Models Reason About Anatomy? Probing the Spatial Inductive Biases of Learned Visual Representations

Interpreting a CT scan means comparing structures on either side, judging how far apart organs sit, and knowing where each one belongs. Medical vision encoders are evaluated on diagnostic accuracy, or through assembled multimodal systems where a failure is hard to attribute, so it remains unclear whether their representations support any of this. We construct SPAR-Bench, eight probes over multi-organ abdominal CT that separate coordinate localization, relational reasoning, and spatial queries, and apply them to five architectural configurations and three medical foundation models, frozen and finetuned. Probes that ask for a comparison within the slice stay at chance, and neither pretraining scale, finetuning, nor architecture closes the gap. Probes that appear solved in domain fall to chance under zero-shot transfer, indicating that their accuracy reflects recall of canonical anatomy rather than computation over the image. Reading the same frozen features with a pooled head rather than the full set of tokens moves relational recovery from 0.7% to 67.8%, so pooled probing understates what a representation holds. Questions the encoders answer well are answered at chance by four open-weight MLLMs. Our results suggest these encoders carry a map of where organs usually lie, and little of the machinery for comparing structures within a particular patient. Code and data will be available at https://spar-bench.github.io.

Naren Akash, Neeraja Ramanan

Center for Visual Information Technology, IIIT Hyderabad, Hyderabad, India · Center for AI and Innovation, AIG Hospitals, Hyderabad, India

Medical Imaging Foundation Models Medical Image Analysis Medical Image Representation Learning Spatial Reasoning Benchmarks Medical Image Localization

Ask PDF

Aug 28, 2026

Beyond Representation Learning: A Systematic Study of Joint-Embedding Predictive Generation for 3D Brain MRI

Joint-embedding predictive architectures (JEPAs) have primarily been developed for self-supervised representation learning. Denoising JEPA (D-JEPA) recently demonstrated strong generative capabilities on natural images, yet the applicability to 3D medical imaging remains unexplored. Building on the D-JEPA framework, we present Med-D-JEPA, a systematic adaptation and evaluation of joint-embedding predictive generation for 3D brain MRI. Med-D-JEPA operates on continuous latent tokens produced by a 3D KL-regularized adversarial variational autoencoder, and combines masked context prediction, representation-level alignment, per-token diffusion, and iterative next-set-of-token sampling. We evaluate unconditional and class-conditional generation quality on BraTS2019 and OASIS-1 datasets; downstream classification utility; and preliminary whole-tumor segmentation on BraTS2020. Across different generation settings, Med-D-JEPA achieves superior or competitive performance compared to several strong baselines on fidelity and diversity metrics. Compared to training with real samples, Med-D-JEPA-based synthetic pretraining improves classification AUC from 0.63 to 0.85 on BraTS2019 and from 0.78 to 0.87 on OASIS-1. In the segmentation study, pretraining on Med-D-JEPA samples improves Dice from 0.74 to 0.80 and reduces HD95 from 13.40 to 9.56 mm. These findings establish joint-embedding predictive generation as a promising direction for 3D medical image synthesis and encourage further research in this direction.

Meng Zhou, Wenhao You, Yuxing Chen +1

University of Toronto · University of Waterloo · University of Alberta +1

Joint-Embedding Predictive Architecture Medical Image Synthesis 3D Medical Imaging Synthetic Data Pretraining 3D Diffusion Models

Ask PDF