FlowNeg: GFlowNet-Guided Diverse Hard Negative Sampling for Knowledge Graph Embedding

Ibne Farabi Shihab; Naoshin Anzum Hridi; Joyanta Jyoti Mondal

cs.LGAug 24, 2026

FlowNeg: GFlowNet-Guided Diverse Hard Negative Sampling for Knowledge Graph Embedding

Authors: Ibne Farabi Shihab, Naoshin Anzum Hridi, Joyanta Jyoti Mondal

Organizations: Department of Computer Science, Iowa State University, USA · Department of Computer Science and Engineering, BRAC University, Bangladesh · Department of Computer and Information Sciences, University of Delaware, USA

Abstract

Negative sampling determines whether a knowledge graph embedding (KGE) model learns from informative counterexamples or wastes updates on implausible corruptions. Uniform negatives are diverse but easy, whereas hard-negative miners concentrate on few entities and collide more with held-out positives. We introduce FlowNeg, a context-conditioned hierarchical generative flow network that amortizes reward-proportional sampling without normalizing a composite reward over the entity set: given a positive triple and corruption side, it selects a type, then an entity. Its terminal reward combines bounded model-based hardness with a training-only structural score for held-out-positive collision, over a relation-specific type-compatible support. We derive the reward, specialize standard trajectory balance, and bound multiplicatively how residual imbalance perturbs terminal and mode probability. Across a descriptive five-seed grid of five architectures and five benchmarks, FlowNeg has higher mean MRR than EMU and than IF-NS in 24 of 25 cells ( $+0.0172$ and $+0.0160$ on average). A separate 15-seed FB15k-237/RotatE control fixing negative count, diagnostic budget, and compute gives FlowNeg $0.359\pm0.001$ MRR against $0.346\pm0.002$ for EMU, with near-uniform fixed-partition diversity, high gradient informativeness, and low collision. The evidence supports mode-covering negative generation without treating structural similarity as an open-world truth oracle.

Figures & tables

Panel of Figure 1 — Figure 1: FlowNeg’s training loop. Both policies and the learned normalizer condition on the full context $\bm{x}$ ; the sampler chooses a type, then an entity, and generated entities are mixed with $\alpha=0.1$ uniform exploration. The KGE is updated every step, whereas the GFlowNet is updated once per $m=5$ KGE steps after a $W=50$ epoch warm-up.

Model	FB15k-237	WN18RR	YAGO3-10	CoDEx-L	Hetionet
TransE	.338/.321/.319	.259/.245/.246	.498/.470/.474	.294/.273/.278	.335/.311/.316
RotatE	.357/.345/.343	.494/.481/.480	.548/.527/.529	.324/.308/.310	.361/.341/.343
ComplEx	.348/.336/.334	.487/.489/.474	.533/.514/.512	.314/.298/.295	.348/.329/.326
ConvE	.354/.340/.341	.458/.444/.445	.512/.491/.493	.305/.286/.288	.339/.319/.322
SimKGC	.362/.348/.349	.503/.491/.492	.559/.534/.560	.333/.315/.316	.372/.350/.351

Table 1: Descriptive test MRR over five paired seeds in the unified model–dataset grid. Each cell is FlowNeg / EMU / IF-NS; complete standard deviations and ranking metrics are in Appendix H .

	vs. EMU		vs. IF-NS
Setting	$\Delta$ MRR	W	$\Delta$ MRR	W
RotatE/FB15k-237	$+0.013$	15	$+0.018$	15
RotatE/WN18RR	$+0.013$	15	$+0.014$	15
RotatE/YAGO3-10	$+0.021$	15	$+0.019$	15
RotatE/CoDEx-L	$+0.016$	15	$+0.014$	15
RotatE/Hetionet	$+0.020$	15	$+0.018$	15

Table 2: Nine-setting

15

-seed paired inference;

\Delta

MRR is FlowNeg minus baseline, with wins W out of 15. The

16

contrasts not marked

\dagger

all attain the tied minimum Holm-adjusted

p=0.0011

. The two disclosed non-rejections (

\dagger

) are ComplEx/WN18RR versus EMU (

p=0.292

) and SimKGC/YAGO3-10 versus IF-NS (

p=0.781

Sampler	Final MRR	Final H@10	NDS $\uparrow$	HPC (%) $\downarrow$	GI $\uparrow$
Uniform	.329 $\pm$ .002	.523 $\pm$ .003	48.1 $\pm$ .2	.3 $\pm$ .0	.13 $\pm$ .01
Self-Adv	.339 $\pm$ .002	.535 $\pm$ .003	13.4 $\pm$ .5	2.0 $\pm$ .1	.59 $\pm$ .02
KBGAN	.333 $\pm$ .003	.529 $\pm$ .004	9.1 $\pm$ .6	3.2 $\pm$ .2	.72 $\pm$ .03
NSCaching	.334 $\pm$ .002	.530 $\pm$ .003	15.8 $\pm$ .4	2.6 $\pm$ .1	.65 $\pm$ .02
SANS	.331 $\pm$ .002	.527 $\pm$ .003	22.0 $\pm$ .4	1.4 $\pm$ .1	.49 $\pm$ .02
EMU	.346 $\pm$ .002	.544 $\pm$ .003	19.4 $\pm$ .3	1.8 $\pm$ .1	.83 $\pm$ .02

Table 3: Matched-

k

FB15k-237/RotatE control, 15 paired seeds (mean

\pm

s.d.). Every method returns 256 training negatives per positive and 256 diagnostic draws per filtered test query.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning
$\mathcal{G}=(\mathcal{E},\mathcal{R},\mathcal{T})$	entities, relations, and observed triples
$\bm{x}=(h,r,t,s)$	positive triple and corruption side
$f_{\theta}(h,r,t)$	KGE score with parameters $\theta$
$q(\hat{e}\mid\bm{x})$	context-conditioned negative proposal
$P_{F},P_{B}$	GFlowNet forward and backward policies
$Z_{\phi}(\bm{x})$	context-conditioned partition function

Appendix

Table 4: Notation used in the method and analysis.

Step	Operation
1	From $\mathcal{T}_{\mathrm{tr}}$ , build and freeze $\{\mathcal{E}_{\tau}\}$ , $\mathcal{A}_{r}^{s}$ , and $\mathcal{N}_{r}^{s}(e)$ .
2	For epochs $1,\ldots,W$ , update $\theta$ with uniform negatives using Equation ( 2 ).
3	For each later mini-batch and observed $(h,r,t)$ , draw side $s$ , form $\bm{x}=(h,r,t,s)$ , and repeat Steps 4–5 for $i=1,\ldots,k$ .
4	Sample $\tau_{i}\sim P_{F}(\cdot\mid\bm{x})$ and $\hat{e}_{i}\sim P_{F}(\cdot\mid\tau_{i},\bm{x})$ .
5	With probability $\alpha$ , replace $\hat{e}_{i}$ by a uniform type-valid draw.
6	Update $\theta$ on the mixed negatives using Equation ( 2 ).

Appendix

Table 5: FlowNeg joint training. Indentation indicates loop scope.

Method	H	D	C	G
Uniform ( Bordes et al., 2013 )	–	$\checkmark$	$\circ$	–
Self-Adv ( Sun et al., 2019 )	$\circ$	–	–	–
KBGAN ( Cai and Wang, 2018 )	$\checkmark$	–	–	$\checkmark$
NSCaching ( Zhang et al., 2019 )	$\checkmark$	$\circ$	–	–
SANS ( Ahrabian et al., 2020 )	$\circ$	$\circ$	$\circ$	–
EMU ( Takamoto et al., 2025 )	$\checkmark$	–	–	$\checkmark$

Appendix

Table 6: Mechanism-level comparison. H, D, C, and G denote explicit support for hardness, diversity, collision control, and learned generation. A circle denotes indirect or partial support.

Audit	Result
Tail identity sensitivity ( $t$ varied)	10000/10000
Head identity sensitivity ( $h$ varied)	10000/10000
Corruption-side sensitivity ( $s$ flipped)	10000/10000
Type-policy gradient reachability (4 comp.)	1000/1000
Entity-policy gradient reachability (4 comp.)	1000/1000
Normalizer gradient reachability (4 comp.)	1000/1000

Appendix

Table 7: Unit and integration audit over

10{,}000

constructed context pairs and

1{,}000

training batches.

Dataset	Signed	Mean $\|\cdot\|$	Med. $\|\cdot\|$	p95	p99
FB15k-237	$+0.004$	0.061	0.043	0.152	0.247
WN18RR	$+0.002$	0.054	0.038	0.136	0.221
YAGO3-10	$-0.006$	0.067	0.047	0.166	0.271
CoDEx-L	$+0.005$	0.063	0.044	0.158	0.259
Hetionet	$-0.003$	0.059	0.041	0.149	0.241

Appendix

Table 8: Held-out trajectory-balance residuals

\delta_{\mathrm{TB}}

on final checkpoints (contexts not used for sampler updates). Diagnostics of approximate balance, not a certificate.

Variant	MRR	Mean $\|\delta\|$	p95 $\|\delta\|$	HPC	NDS ind
Full $(h,r,t,s)$	$.359$ $\pm$ .001	0.061	0.152	0.50	43.8
Omit replaced entity	$.351$ $\pm$ .002	0.194	0.438	0.84	39.6
Pair-only $Z_{\phi}$	$.354$ $\pm$ .002	0.176	0.401	0.73	41.0

Appendix

Table 9: Context-conditioning negative controls on the

15

-seed FB15k-237/RotatE protocol.

Dataset	$\|\mathcal{E}\|$	$\|\mathcal{R}\|$	Train	Validation	Test	Domain
FB15k-237	14,541	237	272,115	17,535	20,466	general
WN18RR	40,943	11	86,835	3,034	3,134	lexical
YAGO3-10	123,182	37	1,079,040	5,000	5,000	general
CoDEx-L	77,951	69	551,982	30,622	30,622	general
Hetionet	47,031	24	1,690,693	56,356	56,356	biomedical

Appendix

Table 10: Benchmark statistics. Validation and test sizes refer to the standard splits used for filtered evaluation.

Method	Selected configuration	Configurations evaluated
Uniform	relation-specific Bernoulli head/tail corruption	1
Self-Adv	1,024-candidate uniform pool; temperature 1.0; 256 weighted draws without replacement	6
KBGAN	RotatE generator; 1,024-candidate pool; 256 categorical draws with replacement; deduplication and uniform backfill	8
NSCaching	cache size 1,024; refresh every 50 optimizer steps; 256 outputs	9
SANS	two-hop structural pool; 0.8 structural and 0.2 uniform backoff	6
EMU	four mutation steps; step size 0.05; 1,024 candidates; 256 outputs	9

Appendix

Table 11: Negative-generation settings for the controlled FB15k-237/RotatE study. Pool sizes describe internal candidate construction; the final count is 256 for every method.

Model	Sampler	FB15k-237	WN18RR	YAGO3-10	CoDEx-L	Hetionet
TransE	FlowNeg	.338 $\pm$ .002	.259 $\pm$ .003	.498 $\pm$ .002	.294 $\pm$ .002	.335 $\pm$ .002
	EMU	.321 $\pm$ .002	.245 $\pm$ .003	.470 $\pm$ .002	.273 $\pm$ .002	.311 $\pm$ .002
	IF-NS	.319 $\pm$ .002	.246 $\pm$ .002	.474 $\pm$ .003	.278 $\pm$ .002	.316 $\pm$ .002
RotatE	FlowNeg	.357 $\pm$ .001	.494 $\pm$ .002	.548 $\pm$ .001	.324 $\pm$ .002	.361 $\pm$ .001
	EMU	.345 $\pm$ .002	.481 $\pm$ .002	.527 $\pm$ .002	.308 $\pm$ .002	.341 $\pm$ .002
	IF-NS	.343 $\pm$ .002	.480 $\pm$ .002	.529 $\pm$ .002	.310 $\pm$ .002	.343 $\pm$ .002

Appendix

Table 12: Unified-grid test MRR, mean

\pm

standard deviation over five paired seeds.

		FB15k-237				WN18RR
Model	Sampler	MRR	H@1	H@3	H@10	MRR	H@1	H@3	H@10
TransE	Uniform	.294 $\pm$ .002	.205 $\pm$ .001	.327 $\pm$ .002	.465 $\pm$ .003	.226 $\pm$ .003	.015 $\pm$ .001	.400 $\pm$ .004	.501 $\pm$ .004
	Self-Adv	.312 $\pm$ .002	.220 $\pm$ .002	.345 $\pm$ .003	.487 $\pm$ .003	.237 $\pm$ .002	.028 $\pm$ .001	.412 $\pm$ .003	.518 $\pm$ .003
	NSCaching	.318 $\pm$ .003	.226 $\pm$ .002	.352 $\pm$ .002	.492 $\pm$ .004	.241 $\pm$ .003	.032 $\pm$ .002	.418 $\pm$ .004	.525 $\pm$ .004
	FlowNeg	.338 $\pm$ .002	.245 $\pm$ .001	.374 $\pm$ .002	.517 $\pm$ .003	.259 $\pm$ .003	.048 $\pm$ .001	.438 $\pm$ .003	.551 $\pm$ .003
RotatE	Uniform	.328 $\pm$ .001	.237 $\pm$ .002	.363 $\pm$ .002	.522 $\pm$ .003	.473 $\pm$ .002	.427 $\pm$ .003	.491 $\pm$ .002	.564 $\pm$ .003

Appendix

Table 13: Filtered ranking metrics on FB15k-237 and WN18RR, mean

\pm

standard deviation over five seeds.

		YAGO3-10		CoDEx-L		Hetionet
Model	Sampler	MRR	H@10	MRR	H@10	MRR	H@10
TransE	Uniform	.442 $\pm$ .002	.610 $\pm$ .003	.251 $\pm$ .003	.402 $\pm$ .004	.288 $\pm$ .003	.452 $\pm$ .004
	EMU	.470 $\pm$ .002	.638 $\pm$ .003	.273 $\pm$ .002	.429 $\pm$ .003	.311 $\pm$ .002	.479 $\pm$ .003
	FlowNeg	.498 $\pm$ .002	.669 $\pm$ .003	.294 $\pm$ .002	.457 $\pm$ .003	.335 $\pm$ .002	.510 $\pm$ .003
RotatE	Uniform	.493 $\pm$ .001	.658 $\pm$ .002	.283 $\pm$ .002	.439 $\pm$ .003	.312 $\pm$ .002	.487 $\pm$ .003
	EMU	.527 $\pm$ .002	.690 $\pm$ .003	.308 $\pm$ .002	.468 $\pm$ .003	.341 $\pm$ .002	.519 $\pm$ .003

Appendix

Table 14: Filtered MRR and Hits@10 on YAGO3-10, CoDEx-L, and Hetionet, mean

\pm

standard deviation over five seeds.

Model	Dataset	FlowNeg	EMU	IF-NS
RotatE	FB15k-237	$.359$ $\pm$ .001	$.346$ $\pm$ .002	$.341$ $\pm$ .002
RotatE	WN18RR	$.495$ $\pm$ .002	$.482$ $\pm$ .002	$.481$ $\pm$ .002
RotatE	YAGO3-10	$.549$ $\pm$ .001	$.528$ $\pm$ .002	$.530$ $\pm$ .002
RotatE	CoDEx-L	$.325$ $\pm$ .002	$.309$ $\pm$ .002	$.311$ $\pm$ .002
RotatE	Hetionet	$.362$ $\pm$ .001	$.342$ $\pm$ .002	$.344$ $\pm$ .002
TransE	YAGO3-10	$.499$ $\pm$ .002	$.471$ $\pm$ .002	$.475$ $\pm$ .003

Appendix

Table 15: Final-budget MRR (mean

\pm

sd over

15

paired seeds) in the nine-setting panel.

Seed	Uniform	EMU	IF-NS	FlowNeg selected	Selected epoch	FlowNeg final
0	.326	.343	.338	.357	271	.360
1	.327	.345	.339	.358	278	.359
2	.330	.347	.342	.359	274	.359
3	.331	.348	.343	.360	282	.357
4	.329	.346	.341	.359	269	.358
5	.330	.347	.342	.360	276	.360

Appendix

Table 16: Seed-level FB15k-237/RotatE MRR in the controlled study. Selected epoch applies to the validation-selected FlowNeg checkpoint; final MRR uses epoch 300.

Quantity	Result
positive mean differences	25/25
Holm-adjusted $p\leq.05$	23/25
mean cell difference	+.0395
median cell difference	+.0418
minimum / maximum	+.0178 / +.0582
null or negative means	0

Appendix

Table 17: Summary of the pre-specified 15-seed FlowNeg–Uniform analysis.

Variant	MRR	H@10	NDS	HPC (%)	GI
FlowNeg	.357	.556	45.3	.5	.89
w/o collision discount	.341	.538	43.1	2.7	.93
w/o type compatibility	.348	.545	38.6	.6	.87
w/o hardness	.308	.488	47.1	.2	.18
flat entity selection	.350	.548	31.2	.5	.85
no warm-up ( $W{=}0$ )	.346	.541	39.7	.7	.81

Appendix

Table 18: Five-seed FlowNeg ablation on FB15k-237/RotatE. One reward factor or architectural component changes in each row. This profile is distinct from the 15-seed control in Table 3 .

Sampler	NDS ind	UER	ESS	T10 $\downarrow$	IPD
Uniform	46.9	0.887	204.8	0.061	0.912
Self-Adv	15.1	0.327	58.7	0.286	0.483
KBGAN	10.7	0.246	41.2	0.371	0.421
NSCaching	17.4	0.351	65.3	0.259	0.512
SANS	24.8	0.446	89.6	0.203	0.604
EMU	20.6	0.384	73.8	0.237	0.557

Appendix

Table 19: Independent diversity on the

15

-seed FB15k-237/RotatE protocol. NDS ind , unique-entity ratio (UER), inverse-Simpson support (ESS), top-

10

mass (T10

\downarrow

), independent pairwise distance (IPD).

Model	Dataset	FlowNeg	EMU	IF-NS
RotatE	FB15k-237	43.8/.792	20.6/.384	24.2/.428
RotatE	WN18RR	44.6/.814	22.1/.412	25.0/.447
RotatE	YAGO3-10	42.7/.765	18.9/.351	22.8/.402
RotatE	CoDEx-L	41.9/.748	19.7/.369	23.4/.416
RotatE	Hetionet	43.3/.781	20.2/.377	24.1/.433
TransE	YAGO3-10	41.8/.756	18.4/.344	22.0/.395

Appendix

Table 20: Consistency of independent NDS / unique-entity ratio over the nine-setting panel.

Dataset	$\operatorname{ECE}_{\mathrm{HPC}}$
FB15k-237	.024
WN18RR	.018
YAGO3-10	.031
CoDEx-L	.027
Hetionet	.022

Appendix

Table 21: Ten-bin calibration error for predicting held-out-positive collision.

Dataset	Prev. (%)	AUROC	AUPRC	$\times$ prev	Raw ECE
FB15k-237	0.50	0.887	0.081	16.2	0.024
WN18RR	0.35	0.841	0.052	14.9	0.018
YAGO3-10	0.70	0.902	0.118	16.9	0.031
CoDEx-L	0.85	0.873	0.126	14.8	0.027
Hetionet	0.60	0.861	0.092	15.3	0.022

Appendix

Table 22: Collision prevalence, discrimination, and calibration (mean

\pm

cluster-bootstrap sd).

Dec.	FB15k	WN18	YAGO	CoDEx	Hetio.
1	0.30/0.00	0.20/0.00	0.40/0.00	0.40/0.05	0.30/0.00
2	0.50/0.05	0.40/0.00	0.70/0.10	0.60/0.10	0.50/0.05
3	0.80/0.10	0.60/0.05	1.00/0.15	0.90/0.20	0.80/0.10
4	1.10/0.15	0.80/0.10	1.40/0.20	1.30/0.30	1.10/0.20
5	1.50/0.25	1.10/0.15	1.90/0.30	1.80/0.40	1.50/0.30
6	2.00/0.35	1.50/0.20	2.60/0.40	2.40/0.60	2.00/0.40

Appendix

Table 23: Ten equal-frequency reliability bins: mean raw score / observed collision rate (%).

Method	Time (h)	Peak VRAM (GB)	MRR
Uniform	2.1	4.2	.328
Self-Adv	2.3	4.4	.338
NSCaching	3.1	5.1	.332
KBGAN	5.8	7.8	.331
EMU	3.4	4.9	.345
IF-NS	8.7	12.4	.340

Appendix

Table 24: Method-profiled full runs on FB15k-237/RotatE using four NVIDIA A100 80GB GPUs.

	Final checkpoint			Validation-selected
Method	2.1 h	3.4 h	4.2 h	2.1 h	3.4 h	4.2 h
Uniform	.328	.329	.329	.329	.330	.330
Self-Adv	.333	.338	.339	.335	.340	.340
KBGAN	.319	.330	.333	.321	.332	.334
NSCaching	.326	.334	.334	.328	.336	.336
SANS	.325	.330	.331	.327	.332	.333

Appendix

Table 25: Test MRR at equal wall-clock budgets on FB15k-237/RotatE (15 paired seeds): the final complete checkpoint written by each deadline, and the best validation-selected checkpoint available by that deadline.

KG Embedding

Knowledge graph embedding (KGE) represents entities and relations as vectors or structured geometric objects for prediction and reasoning.

See all 56 papers

How Many and How Much? Contrastive Learning with Negative Samples for Knowledge Graph Embedding
Oct 27, 2026Sooho Moon, Yunyong KoKG Embedding Contrastive Learning
Chung-Ang University Seoul, Korea
PyKEEN-NSX: A Modular Framework for Static, Dynamic and Schema-Aware Negative Sampling in PyKEEN
Aug 31, 2026Ivan Diliso, Nicola Fanizzi, Claudia d'AmatoKG Embedding KG Completion
1Dipartimento di Informatica, University of Bari Aldo Moro, Bari, Italy
Boosting Knowledge Graph Foundation Models via Enhanced Negative Sampling
May 26, 2026Yinan Liu, Wenjin Xu, Zhiyuan Zha +2KG Embedding Graph Foundation Models
Northeastern University Shenyang, China

Oct 27, 2026J/K move · Enter open · S save

How Many and How Much? Contrastive Learning with Negative Samples for Knowledge Graph Embedding

Sooho Moon, Yunyong Ko

Chung-Ang University Seoul, Korea

KG Embedding Contrastive Learning

Open paper AskPDF arXiv

Knowledge graph embedding (KGE), a fundamental technique for knowledge-centric Web and KG-augmented AI applications, is commonly trained through contrastive learning with negative samples to learn discriminative representations. Despite its empirical success, two crucial questions regarding negative samples remain underexplored: (Q1) how many negative samples should be used for each positive sample and (Q2) how much should each negative sample contribute to learning? To answer these questions, we propose RADAR, a novel contrastive learning framework for KGE models with two effective strategies, respectively: (1) Risk-aware negative budget allocation quantifies false-negative sampling risk by modeling an open-world KG as a stochastic partial observation of its closed-world counterpart and adaptively assigns a negative budget to each positive sample based on its risk; (2) Distribution-calibrated negative weighting assigns negative weights by calibrating the weighting strength for each positive based on two complementary characteristics of the negative-score distribution: dispersion and concentration. Extensive experiments on 3 real-world KGs with 4 KGE models reveal that RADAR achieves the best KGE performance on average across all model--KG combinations and reduces sampled false negatives by over 90% compared with conventional fixed-budget methods.

Abstract

KG Embedding

Explore similar work

How Many and How Much? Contrastive Learning with Negative Samples for Knowledge Graph Embedding

PyKEEN-NSX: A Modular Framework for Static, Dynamic and Schema-Aware Negative Sampling in PyKEEN

Boosting Knowledge Graph Foundation Models via Enhanced Negative Sampling