Witness Overlap: Directional Provenance Inside Open-Weight Model Families

Siyuan Li; Haoxuan Zeng; Xin Luo; Fernando Jia; Florence Li; Zhengyang Geng; Zico Kolter; Tai Sing Lee

cs.AISep 24, 2026

Witness Overlap: Directional Provenance Inside Open-Weight Model Families

Authors: Siyuan Li, Haoxuan Zeng, Xin Luo, Fernando Jia, Florence Li, Zhengyang Geng, Zico Kolter, Tai Sing Lee,

Organizations: ModelOS · University of Michigan · Carnegie Mellon University

Abstract

Open-weight models are often released, fine-tuned, aligned, merged, and re-released, making provenance audits ask not only whether checkpoints are related, but also which checkpoint came first. Many existing model-provenance methods are designed for a base-known audit setting: given a victim or source model, they test whether a suspect model is related to it. Although these audits are framed as source-to-suspect tests, their underlying evidence is often symmetric, relying on representation similarity, weight similarity, behavioral fingerprints, or correlation statistics. Symmetric pairwise comparisons can detect relatedness, but they cannot by themselves orient relationship between checkpoints A and B. We therefore introduce a local geometric comparison: instead of comparing two checkpoints directly, we add a third same-family checkpoint as a witness and compare the geometry around each candidate endpoint. Direction is inferred by asking which candidate behaves more like a branching parent. Motivated by this idea, and by the empirically observed asymmetry between parent-anchored and child-anchored witness-overlap distributions, we propose Witness Overlap, a prompt-free, training-free white-box test for directional provenance. On 176 LLM checkpoints from 16 families, our one-witness test orients 95.3% of parent-child decisions using Frobenius cosine. We further evaluate root identification, sibling discrimination, generalizations to VLM and diffusion families, and chain-structured ordering. The signal is robust to weight noise and sparse pruning, with a proposed SVD weight reduction variant showing greater robustness than Frobenius cosine.

Figures & tables

Figure 1: Anchor-Conditioned Witness Overlap Across Four Provenance Tasks. Witness Overlap scores each anchor-target-witness triplet by comparing the two update directions from the anchor. (a) When the parent is used as the anchor, the left frequency histogram shows that cosine overlaps between updates to different descendants concentrate near low values; the right schematic illustrates these weakly aligned outgoing directions. The lower-overlap endpoint is predicted as the parent. (b) When a child is used as the anchor, the overlap distribution shifts higher because the directions to the parent and to a sibling share a reverse component toward the parent; the left schematic illustrates this aligned geometry. The higher-overlap endpoint is predicted as the child. (c)–(f) The same anchor-conditioned low/high pattern supports root identification, parent-child orientation, pair-type discrimination, and single-chain ordering: select the lowest-scoring candidate root, compare endpoint anchor scores, separate low-high from high-high patterns, and sort checkpoints by average overlap relative to a known base to identify the chain.

Method	LLM	VLM	Diffusion
Witness Overlap, Frobenius cosine (ours)	93.8%	100.0%	80.0%
Witness Overlap, SVD top- $k{=}16$ (ours)	100.0%	100.0%	80.0%
L2 medoid	81.3%	75.0%	60.0%
Neural Phylogeny (norm) ( Yu and Wang, 2025 )	18.8%	0.0%	20.0%
MoTHer ( Horwitz et al., 2025 )	18.8%	25.0%	60.0%

Table 1: Root identification. Accuracy of selecting the base checkpoint within each family. Relatedness methods such as REEF ( Zhang et al., 2025 ) , LLM DNA ( Wu et al., 2026 ) , and black-box provenance tests ( Nikolic et al., 2025 ) are omitted because they use symmetric or non-orienting pairwise scores and therefore cannot directly identify the root within a same-family set. Witness Overlap chooses the checkpoint with the lowest mean overlap when used as an anchor. The L2 medoid selects the checkpoint with the lowest mean pairwise Euclidean distance. Neural Phylogeny uses its parameter-norm endpoint rule. MoTHer uses its kurtosis-based directed tree recovery.

Method	LLM SFT	LLM RLHF	LLM Comb.	VLM SFT	Diffusion SFT
Witness Overlap, Frobenius cosine (ours)	96.2%	91.4%	95.3%	100.0%	85.0%
Witness Overlap, SVD top- $k{=}16$ (ours)	95.2%	86.4%	93.5%	87.5%	90.0%
Neural Phylogeny (norm) ( Yu and Wang, 2025 )	79.2%	66.7%	76.9%	50.0%	50.0%
MoTHer kurtosis ( Horwitz et al., 2025 )	79.2%	53.3%	74.4%	68.8%	70.0%

Table 2: Parent-child direction accuracy at

N_{w}{=}1

. The LLM columns use the

16

-family open-weight LLM collection. VLM and Diffusion report four VLM families and five Diffusion families. Witness rows use one same-family witness per decision. Scalar baselines are witness-free.

Method	LLM		VLM		Diffusion
	AUROC	Bal. acc	AUROC	Bal. acc	AUROC	Bal. acc
Frobenius cosine	0.898	82.4%	0.903	79.9%	0.855	80.3%
SVD top- $k{=}16$	0.780	68.6%	0.715	54.9%	0.855	53.3%

Table 3: Pairwise-type discrimination at

N_{w}{=}1

, leave-one-family-out. A logistic regressor on

\min\{s_{A}(A,B;W),s_{B}(A,B;W)\}

distinguishes parent–child from sibling pairs using one witness. Thresholds are selected only on training families. Bal. acc. denotes balanced accuracy.

Panel of Figure 3 — Figure 3: Single-witness Frobenius-cosine margin and $\rho/\lambda$ scatter plot and distribution. Panel (a) plots Frobenius-cosine margin at $N_{w}{=}1$ against $\rho/\lambda$ on a symlog scale, and the dashed line marks the sufficient-condition boundary $\rho/\lambda=1/2$ from Proposition 4.1 . Panel (b) shows the single-witness margin distribution over all LLM triplets. SVD has $1442/1542$ positive margins with median $+0.31$ , and Frobenius cosine has $1469/1542$ positive margins with median $+0.57$ . Panel (c) shows the empirical distribution of $\rho/\lambda$ for the same Frobenius-cosine triplets, with most values below $0.5$ .

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	Value
Precision	99.32%
Recall	91.25%
Specificity	99.96%
F1	95.11%
Accuracy	99.41%

Appendix

Table 4: Black-box outputs and multiple hypothesis testing [ Nikolic et al., 2025 ] ’s performance on our 176 LLM models. The evaluation contains 160 true root-derived pairs and 2,400 unrelated child-root controls (TP=146, FN=14, FP=1, TN=2,399).

Metric	Root ID	Random-one direction	All-witness direction
Cosine	$22/22\ (100\%)$	$2{,}145/2{,}200\ (97.50\%)$	$220/220\ (100\%)$
SVD	$22/22\ (100\%)$	$2{,}114/2{,}200\ (96.09\%)$	$220/220\ (100\%)$

Appendix

Table 5: Stage-1 false-positive stress test. One architecture-compatible checkpoint from the Llama-3-8B or Mistral-7B family is injected into the other family. Root ID is evaluated over the

22

contaminated pools. Direction is evaluated over

220

true edges with ten eligible witness choices per edge.

Family	Triplets	median $\rho$	median $\lambda$	median $\rho/\lambda$
DeepSeek-R1-Distill-Qwen-1.5B	72	0.0011	1.0000	4.86e-04
Gemma-3-4B-it	42	1.98e-04	1.0021	4.34e-04
GPT-2	90	0.0087	1.0002	0.0056
GPT-2 Medium	90	0.0026	1.0000	0.0022
GPT-2 XL	56	6.44e-04	1.0015	3.15e-04
Llama-3.2-1B-Instruct	182	3.32e-04	1.0002	2.14e-04

Appendix

Table 6: Per-family single-witness geometry. Each row summarizes all LLM

N_{w}{=}1

parent-child-witness triplets in one family. Here

i

is the evaluated child update,

j

is the witness update,

\rho=C(p;i,j)

, and

\lambda=\|u_{i}\|/\|u_{j}\|

. The final column reports the median of the triplet-level ratio

\rho/\lambda

, which is the quantity compared with

1/2

in Proposition 4.1 .

Panel of Figure 4 — Figure 4: Cross-domain $N_{w}$ -witness ablation. Here $N_{w}$ is the number of same-family witnesses averaged before making a decision. The main result uses $N_{w}{=}1$ ; larger $N_{w}$ tests whether pooling witnesses stabilizes the same directional signal on VLM and Diffusion families.

Merge	Root ID	$N_{w}{=}1$ direction	All-witness direction
TIES $0.50/0.50$	$93.75\%/87.50\%$	$97.03\%/96.95\%$	$100.00\%/100.00\%$
TIES $0.25/0.75$	$93.75\%/93.75\%$	$97.03\%/96.88\%$	$100.00\%/100.00\%$
SLERP $0.50/0.50$	$93.75\%/87.50\%$	$96.09\%/96.41\%$	$97.50\%/100.00\%$
SLERP $0.25/0.75$	$81.25\%/87.50\%$	$95.70\%/96.56\%$	$98.13\%/100.00\%$

Appendix

Table 7: Same-family merge evaluation. We merge two descendants that share a root using TIES or SLERP. Root ID reports accuracy over the resulting four-checkpoint candidate sets. Direction reports the accuracy of placing the merged checkpoint on the descendant side of the root using one or all eligible witnesses. Each entry gives Frobenius cosine / SVD top-

k{=}16

Family	Metric	Root ID	$N_{w}{=}1$ direction	All-witness direction
Llama-3.2-1B-Instruct	Cosine	$1/1$	$182/182$	$14/14$
	SVD	$1/1$	$179/182$	$14/14$
Qwen2.5-0.5B	Cosine	$1/1$	$169/182$	$14/14$
	SVD	$1/1$	$159/182$	$14/14$
Qwen3-0.6B-Base	Cosine	$1/1$	$72/72$	$9/9$
	SVD	$1/1$	$68/72$	$9/9$

Appendix

Table 8: All-checkpoint INT8 quantization. The same symmetric per-tensor signed INT8 quantize–dequantize rule is applied to every checkpoint in each family. Direction is evaluated with one or all eligible witnesses.

Setting	Cosine, single / all witnesses	SVD, single / all witnesses
Clean control	$752/800\ (94.00\%)$ / $64/65$	$731/800\ (91.38\%)$ / $65/65$
Child-only INT8	$794/800\ (99.25\%)$ / $65/65$	$794/800\ (99.25\%)$ / $65/65$

Appendix

Table 9: Child-only INT8 quantization. The parent and witness remain clean while only the queried child is quantized. The clean control uses the same five families and triplets.

Method	Anchor role	LLM			VLM			Diffusion
		Mean	Median	Std	Mean	Median	Std	Mean	Median	Std
Frobenius cosine	parent	0.0232	0.0025	0.0482	0.0350	0.0009	0.0546	0.0950	0.0294	0.1615
Frobenius cosine	child	0.5527	0.5678	0.2704	0.5140	0.4647	0.3056	0.4278	0.3845	0.2582
SVD top- $k{=}16$	parent	0.0725	0.0405	0.0867	0.1034	0.0619	0.0920	0.2454	0.1543	0.2216
SVD top- $k{=}16$	child	0.3961	0.3950	0.1720	0.4181	0.4629	0.2142	0.4537	0.4326	0.2338

Appendix

Table 10: Overlap distribution by anchor role. For each parent–child pair we average the witness overlap over all in-family witnesses, separately for the two anchor choices (parent or child). The table aggregates these per-pair values across the LLM, VLM, and Diffusion evaluations. Both statistics show the same asymmetry: parent-anchor overlap is usually smaller than child-anchor overlap.

Child repository	Witness repository	$\rho$	$\eta$	Tag
INSAIT-Institute/BgGPT-Gemma-3-4B-IT	INSAIT-Institute/MamayLM-Gemma-3-4B-IT-v1.0	0.5841	0.89	high- $\rho$
INSAIT-Institute/MamayLM-Gemma-3-4B-IT-v1.0	INSAIT-Institute/BgGPT-Gemma-3-4B-IT	0.5841	1.12	high- $\rho$
Ssarion/gpt2-multi-news	bipin/malayalam-gpt2	0.0319	97.36	both
Ssarion/gpt2-multi-news	jackoyoungblood/TinyStoriesTest	0.0325	95.51	both
bipin/malayalam-gpt2	jackoyoungblood/TinyStoriesTest	0.9648	0.98	high- $\rho$
crumb/qrstudy-gpt2-2-4	bipin/malayalam-gpt2	0.0028	405.80	high- $\eta$

Appendix

Table 11: Frobenius-cosine single-witness failures. Each row is one misoriented LLM triplet at

N_{w}{=}1

. Here

i

is the child update,

j

is the witness update,

\rho=C(p;i,j)

, and

\eta=\|u_{j}\|/\|u_{i}\|=1/\lambda

. The high-

\rho

and high-

\eta

tags mark the top

10\%

of all cosine triplets, with thresholds

\rho\geq 0.0247

and

\eta\geq 45.16

Model	Rule	Magnitude pruning	$N_{w}{=}1$	$N_{w}{=}2$	$N_{w}{=}3$	$N_{w}{=}\mathrm{all}$
GPT-2	Frobenius cosine	0%	87.8%	98.1%	99.8%	100.0%
GPT-2	Frobenius cosine	2%	87.8%	98.1%	99.8%	100.0%
GPT-2	Frobenius cosine	5%	86.7%	97.8%	99.9%	100.0%
GPT-2	Frobenius cosine	10%	84.4%	98.1%	100.0%	100.0%
GPT-2	Frobenius cosine	30%	80.0%	97.5%	100.0%	100.0%
GPT-2	Frobenius cosine	50%	80.0%	97.5%	100.0%	100.0%

Appendix

Table 12: Child-only pruning stress test. Only the evaluated child checkpoint is pruned, the parent and witness endpoints remain clean. Entries are parent-child direction accuracy after aggregating exactly

N_{w}

same-family witnesses, with

N_{w}{=}\mathrm{all}

using all eligible witnesses.

Model	Rule	Magnitude pruning	$N_{w}{=}1$	$N_{w}{=}2$	$N_{w}{=}3$	$N_{w}{=}\mathrm{all}$
GPT-2	Frobenius cosine	0%	87.8%	98.1%	99.8%	100.0%
GPT-2	Frobenius cosine	2%	87.8%	98.1%	99.8%	100.0%
GPT-2	Frobenius cosine	5%	88.9%	98.1%	99.6%	100.0%
GPT-2	Frobenius cosine	10%	83.3%	92.5%	96.0%	100.0%
GPT-2	Frobenius cosine	30%	62.2%	77.5%	84.9%	100.0%
GPT-2	Frobenius cosine	50%	62.2%	76.1%	76.3%	80.0%

Appendix

Table 13: Half-descendant pruning stress test. A random half of the base model descendants is pruned, the base and remaining descendants remain clean. Entries are parent-child direction accuracy after aggregating exactly

N_{w}

same-family witnesses, with

N_{w}{=}\mathrm{all}

using all eligible witnesses.

Model	Rule	Magnitude pruning	$N_{w}{=}1$	$N_{w}{=}2$	$N_{w}{=}3$	$N_{w}{=}\mathrm{all}$
GPT-2	Frobenius cosine	0%	87.8%	98.1%	99.8%	100.0%
GPT-2	Frobenius cosine	2%	83.3%	90.3%	92.1%	100.0%
GPT-2	Frobenius cosine	5%	73.3%	76.1%	76.3%	70.0%
GPT-2	Frobenius cosine	10%	53.3%	52.5%	49.5%	50.0%
GPT-2	Frobenius cosine	30%	17.8%	17.8%	20.0%	20.0%
GPT-2	Frobenius cosine	50%	17.8%	15.6%	20.0%	20.0%

Appendix

Table 14: All-descendant pruning stress test. All GPT-2 descendants are pruned, while the base checkpoint remains clean. Entries are parent-child direction accuracy after aggregating exactly

N_{w}

same-family witnesses, with

N_{w}{=}\mathrm{all}

using all eligible witnesses.

$\alpha$	GPT-2	TinyLlama-1.1B-Chat-v1.0
0	$79/90=87.8\%$	$54/56=96.4\%$
0.1	$237/270=87.8\%$	$162/168=96.4\%$
0.2	$237/270=87.8\%$	$162/168=96.4\%$
0.5	$240/270=88.9\%$	$168/168=100.0\%$
1	$246/270=91.1\%$	$168/168=100.0\%$
2	$252/270=93.3\%$	$168/168=100.0\%$

Appendix

Table 15: Child Gaussian noise stress test. Only the child checkpoint is perturbed. Larger

\alpha

does not reduce Frobenius-cosine direction accuracy on these two families.

Family	Checkpoint type	HuggingFace repository
DeepSeek-R1-Distill-Qwen-1.5B	base	deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
DeepSeek-R1-Distill-Qwen-1.5B	SFT	jan-hq/AlphaMaze-v0.2-1.5B-SFT
DeepSeek-R1-Distill-Qwen-1.5B	SFT	Menlo/AlphaSpace-1.5B
DeepSeek-R1-Distill-Qwen-1.5B	SFT	thuml/bytesized32-world-model-sft
DeepSeek-R1-Distill-Qwen-1.5B	SFT	lightblue/DeepSeek-R1-Distill-Qwen-1.5B-Multilingual
DeepSeek-R1-Distill-Qwen-1.5B	SFT	avanishd/DeepSeek-R1-Distill-Qwen-1.5B-finetuned-smoltalk-everyday-conversations

Appendix

Table 16: Full LLM checkpoint registry. The table lists the public HuggingFace repositories used for local academic evaluation.

Family	Params	#SFT	#RLHF
DeepSeek-R1-Distill-Qwen-1.5B	1.5B	7	2
Gemma-3-4B-it	4B	6	1
GPT-2	124M	10	0
GPT-2 Medium	355M	8	2
GPT-2 XL	1.5B	8	0
Llama-3.2-1B-Instruct	1B	10	4

Appendix

Table 17: Per-family LLM composition. The table lists each base family, its parameter scale, and the number of SFT and RLHF descendants used in the LLM evaluation.

Domain	Family	Checkpoint type	HuggingFace repository
VLM	PaliGemma2-3B-pt-224	base	google/paligemma2-3b-pt-224
VLM	PaliGemma2-3B-pt-224	SFT	IPEC-COMMUNITY/spatialvla-4b-224-pt
VLM	PaliGemma2-3B-pt-224	SFT	mateoguaman/vamos
VLM	PaliGemma2-3B-pt-224	SFT	mateoguaman/vamos_navigation_only
VLM	PaliGemma2-3B-pt-224	SFT	oliveirabruno01/count_intersection-ft-paligemma2-3b-pt-224
VLM	Qwen2.5-VL-3B-Instruct	base	Qwen/Qwen2.5-VL-3B-Instruct

Appendix

Table 18: Full VLM and Diffusion checkpoint registry. The table lists the public HuggingFace repositories used for the VLM and Diffusion checks.

Domain	Family	Params	#SFT
VLM	PaliGemma2-3B-pt-224	3B	4
VLM	Qwen2.5-VL-3B-Instruct	3B	4
VLM	Qwen2-VL-2B	2B	4
VLM	SmolVLM-500M-Instruct	500M	4
Diffusion	Lumina-Image-2.0	2B	4
Diffusion	Playground v2.5	3.5B	4

Appendix

Table 19: Per-family VLM and Diffusion composition. The table lists each cross-domain family, its domain, parameter scale, and the number of SFT descendants used in the VLM and Diffusion checks.

LLM Auditing

Large language model auditing tests models and deployed systems for bias, safety, privacy, reliability, and provenance using behavioral and internal evidence.

See all 639 papers

Aug 14, 2026

Training Leaves Traces: Centered Residual Signatures for Language Model Lineage Verification

Open-weight language models are fine-tuned, quantized, pruned, and merged, yet their provenance is often undocumented. We study data-free white-box lineage verification: can weights alone reveal whether two compatible model checkpoints share ancestry? Residual training produces a shared identity-aligned component in branch products, so this structure alone cannot establish ancestry. We remove it and compare checkpoint-specific structure across residual blocks, yielding a symmetric lineage score calibrated against independent checkpoints. On residual-MLP and GPT-2 benchmarks, the score separates fine-tuned, LoRA-merged, pruned, and quantized descendants from independent and distilled models (AUROC=1.0), distinguishing weight ancestry from behavioral similarity. Under function-preserving checkpoint laundering experiments, weight-space baselines lose margin or fail; our score remains unchanged and runs 76x faster than the nearest robust baseline on GPT-2. The projection-pairing signal appears across six language-model families and beyond, and a case study correctly identifies 3 related and 7 unrelated LLaMA-2 public checkpoints. Collectively, these results establish a passive, data-free provenance signal for compatible open-weight language-model checkpoints

Aman Singh Thakur, Rayan Khoury

Massachusetts Institute of Technology, USA

Language Model Fingerprinting Open-Weight Language Models Data Provenance in Language Models

Ask PDF

Aug 7, 2026

Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Space

Open-weight large language models (LLMs) are increasingly developed through complex, multi-stage pipelines, leading to intricate lineage relationships that reflect model origin, ownership, and evolution. Understanding these relationships is important for model provenance, governance, and supply-chain integrity. In this work, we investigate the notion of LLM "biometrics" (analogous to human biometrics) to ask whether LLMs exhibit intrinsic fingerprints in weight space alone, without access to input data, that reveal their origin and lineage. We formulate this as a lineage discrimination problem, distinguishing among independent-origin, same-series, and shared-base models. To characterize these relationships, we propose a unified geometric fingerprinting framework that analyzes weight matrices from two complementary perspectives: (i) spectral energy, captured by singular value distributions to encode global magnitude patterns, and (ii) subspace alignment, quantified via subspace deviations to capture directional geometry. Our analysis uncovers a clear hierarchy of structural similarity in weight space: spectral energy reliably distinguishes independently trained models and different model families, while subspace alignment enables fine-grained discrimination among closely related models, including variations in dataset scale and post-training procedures. Extensive experiments on over 110 diverse open-weight LLM pairs demonstrate that weight-space geometry provides a robust and interpretable signal for model lineage, enabling coarse-grained regime separation and fine-grained discrimination within shared-base models.

Yiwei Chen, Bingqi Shang, Sijia Liu

Michigan State University

Language Model Fingerprinting Open-Weight Language Models Weight-Space Analysis

Ask PDF

Apr 7, 2026

AttnDiff: Attention-based Differential Fingerprinting for Large Language Models

Protecting the intellectual property of open-weight large language models (LLMs) requires verifying whether a suspect model is derived from a victim model despite common laundering operations such as fine-tuning (including PPO/DPO), pruning/compression, and model merging. We propose AttnDiff, a data-efficient white-box framework that extracts fingerprints from models via intrinsic information-routing behavior. AttnDiff probes minimally edited prompt pairs that induce controlled semantic conflicts, captures differential attention patterns, summarizes them with compact spectral descriptors, and compares models using CKA. Across Llama-2/3 and Qwen2.5 (3B--14B) and additional open-source families, it yields high similarity for related derivatives while separating unrelated model families (e.g.,

>0.98

vs.\

<0.22

with

M=60

probes). With 5--60 multi-domain probes, it supports practical provenance verification and accountability.

Haobo Zhang, Zhenhua Xu, Junxian Li +3

Zhejiang University of Technology · Binjiang Institute of Zhejiang University · Zhejiang University +2

Language Model Fingerprinting Data Provenance in Language Models Fingerprint Identification

Ask PDF

Abstract

LLM Auditing

Explore similar work

Training Leaves Traces: Centered Residual Signatures for Language Model Lineage Verification

Who Built This Model? Tracing LLM Lineage via Spectral Fingerprints in Weight Space

AttnDiff: Attention-based Differential Fingerprinting for Large Language Models