TIEG-Youpu Solution for NeurIPS 2022 WikiKG90Mv2-LSC
Organizations: TIEG-Youpu Tencent, Shenzhen China · TIEG-Youpu Tsinghua University, Beijing China
Abstract
WikiKG90Mv2 in NeurIPS 2022 is a large encyclopedic knowledge graph. Embedding knowledge graphs into continuous vector spaces is important for many practical applications, such as knowledge acquisition, question answering, and recommendation systems. Compared to existing knowledge graphs, WikiKG90Mv2 is a large scale knowledge graph, which is composed of more than 90 millions of entities. Both efficiency and accuracy should be considered when building graph embedding models for knowledge graph at scale. To this end, we follow the retrieve then re-rank pipeline, and make novel modifications in both retrieval and re-ranking stage. Specifically, we propose a priority infilling retrieval model to obtain candidates that are structurally and semantically similar. Then we propose an ensemble based re-ranking model with neighbor enhanced representations to produce final link prediction results among retrieved candidates. Experimental results show that our proposed method outperforms existing baseline methods and improves MRR of validation set from 0.2342 to 0.2839.
Figures & tables
| Method | Main Parameter Settings | Recall @20000 | Acc. |
| Structure Based Retrieval Models | |||
| PIE_6 | neighbor samples = 6 | 0.6008 | 4.18e-5 |
| PIE_10 | neighbor samples = 10 | 0.6044 | 3.73e-5 |
| PIE_10_upsample | neighbor samples = 10 w/ up sampling | 0.6044 | 3.79e-5 |
| Rule-1 | HT | 0.0568 | 5.46e-3 |
| Rule-2 | TH | 0.0488 | 7.42e-4 |
| Method | Main Parameter Settings | Validation MRR@10 |
|---|---|---|
| TransE-0 | - | 0.214 |
| TransE-1 | batch size = 20480, negative sample size = 20480 | 0.2094 |
| TransE-2 | batch size = 20480, negative sample size = 20480, MPNet and randomly initialized embeddings w/ MLP layer | 0.2114 |
| TransE-3 | neighbor enhanced embeddings and randomly initialized embeddings w/ MLP layer | 0.1877 |
| ComplEx | - | 0.1649 |
| NOTE-0 | - | 0.1561 |
KG Embedding
Knowledge graph embedding (KGE) represents entities and relations as vectors or structured geometric objects for prediction and reasoning.
See all 56 papersSearch-on-Graph-R1: Training Large Language Models to Search Knowledge Graphs with Reinforcement Learning
Search tool rather than having to discover the path itself. Since every call executes against a live Freebase server, the resulting trajectories are grounded in the knowledge graph by construction. On WebQSP, CWQ, and GrailQA, \sogrone{} at 8B surpasses every frozen frontier-LLM system in our comparison and posts the strongest results on CWQ of any system we compare against. It does so using no auxiliary module at inference and no LLM judge during training. Isolating each training stage shows that SFT and RL contribute complementary gains, our approach transfers across model families, and RL learns to reach answers in fewer Search calls than its SFT initialization.