CADEL: A Corpus of Administrative Web Documents for Japanese Entity Linking

Shohei Higashiyama; Masao Ideuchi; Masao Utiyama

cs.CLMar 31, 2026

CADEL: A Corpus of Administrative Web Documents for Japanese Entity Linking

Authors: Shohei Higashiyama, Masao Ideuchi, Masao Utiyama

Organizations: National Institute of Information and Communications Technology

Abstract

Entity linking is the task of associating linguistic expressions with entries in a knowledge base that represent real-world entities and concepts. Language resources for this task have primarily been developed for English, and the resources available for evaluating Japanese systems remain limited. In this study, we develop a corpus design policy for the entity linking task and construct an annotated corpus for training and evaluating Japanese entity linking systems, with rich coverage of linguistic expressions referring to entities that are specific to Japan. Evaluation of inter-annotator agreement confirms the high consistency of the annotations in the corpus, and a preliminary experiment on entity disambiguation based on string matching suggests that the corpus contains a substantial number of non-trivial cases, supporting its potential usefulness as an evaluation benchmark.

Figures & tables

Dataset	Lang.	Data source	KB	#Mentions	Manual
CADEL (Ours)	ja	Web	Wikidata	8,082	$\checkmark$
jawikify Jargalsaikhan et al. (2016)	ja	News	Wikipedia	25,675	$\checkmark$
JWC Murawaki and Mori (2016)	ja	Blog, Twitter, etc.	Wikipedia	8,057	$\checkmark$
Shinra 2023 dataset Sekine et al. (2023)	ja	Wikipedia	Wikipedia	59,429	$\triangle$
WikiANN Pan et al. (2017)	multi	Wikipedia	Wikipedia	1.9M	✗
Mewsli-9 Botha et al. (2020)	multi	Wikinews	Wikidata	34,463	✗

Table 1: Characteristics of EL datasets with Japanese text. For multilingual (multi) datasets, we report the number of mentions only for the Japanese portion. In the Manual Column, “

\checkmark

”, “

\triangle

”, and “ ✗ ” indicate full, partial, and no manual correction, respectively.

Mention								Entry
(R1) Base form		(R2) POS		(R3) Overlap		(R4) Reference		Link type
PROPN: Full name	$\checkmark$	NP: Singular	$\checkmark$	None	$\checkmark$	Direct	$\checkmark$	Exact	$\checkmark$
PROPN: Short name	$\checkmark$	NP: Plural	$\checkmark$	Maximal	$\checkmark$	Anaphoric	$\triangle$	Related	$\checkmark$
PROPN: Extended name	$\checkmark$	Adjective		Intermediate		Metaphoric		–
PROPN: Alias	$\checkmark$	Verb		Minimal		Metonymic	$\checkmark$	–
Numeric/Temporal		Adverb		–		Related		–

Table 2: Overview of CADEL’s design policy, which extends Rosales-Méndez et al. (2019) ’s taxonomy (R1–R4). The subcategory “Related” under R4 follows their taxonomy and is distinct from our “Related” entry link type. PROPN and NP indicate proper noun and noun phrase, respectively.

Tag	Type	ENE category
PER	Person	1.1
LIV	Living things	1.3
ORG	Organization	1.4
LOC	Location	1.5
FAC	Facility	1.6
PRO	Product	1.7 (subset)

Table 3: Entity types for CADEL. NOMINAL is used for non-named mentions across entity types.

#Doc	#Article	#Sent	#Mention	#Cluster
85	160	3,852	8,082	4,049

Table 4: Basic statistics of CADEL.

Type	Mention	Exact link (%)		Related link (%)
NAME	6,939	5,058	(72.9)	702	(10.1)
PER	307	217	(70.7)	1	(0.0)
LIV	8	0	(0.0)	0	(0.0)
ORG	1,319	957	(72.6)	156	(11.8)
LOC	2,596	2,228	(85.8)	297	(11.4)
FAC	1,059	670	(63.3)	173	(16.3)

Table 5: The numbers of mentions and link numbers/ratios of Wikidata entries for each type.

Ranking method	AltLabel	Exact link			Related link
Ranking method	AltLabel	R@1	R@10	R@100	R@1	R@10	R@100
Uniform		0.555	0.756	0.762	0.074	0.099	0.100
Uniform	$\checkmark$	0.565	0.788	0.795	0.086	0.112	0.113
SmallerID		0.693	0.761	0.762	0.080	0.100	0.100
SmallerID	$\checkmark$	0.716	0.794	0.795	0.091	0.113	0.113
MoreWikiBLs		0.733	0.761	0.762	0.098	0.100	0.100

Table 6: Disambiguation performance of string matching for named mentions.

	#Doc	#Art	#Sent	#Mention
Train	39	74	1,607	2,903
Dev	15	31	636	1,354
Test	31	55	1,609	3,825
Total	85	160	3,852	8,082

Table 7: Official data split of CADEL.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Text type	#Docs	#Articles
Public relations magazines	37	79
White and blue papers	21	48
Press conference/meeting records	7	10
Press releases	6	7
Other web pages	8	9
Other PDF documents	6	7

Appendix

Table 8: The numbers of documents/articles for each text media type.

Org.	Domain	#Docs	#Articles
CAO (Bousai)	bousai.go.jp	4	4
CAO (OGB)	ogb.go.jp	1	2
CAO (NPA)	npa.go.jp	2	5
MIC (FDMA)	fdma.go.jp	1	2
MOJ	moj.go.jp	1	3
MOFA	mofa.go.jp	5	5

Appendix

Table 9: The numbers of documents/articles for each source site domain name.

Publication year	#Docs	#Articles
Unknown	6	7
1998	1	1
2017	1	1
2018	1	1
2019	1	5
2020	3	4

Appendix

Table 10: The numbers of documents/articles for each publication year.

PART	CONT	SHARE	PRD_I	DIAC	OTH	VAGUE
497	36	43	20	129	125	43

Appendix

Table 11: Distribution of related link type tags assigned to mentions.

Size	1	2	3	4	5	$\geq$ 6
Clster	2,531	682	336	197	89	214
DM	1.00	1.32	1.57	1.78	2.01	2.73
DNM	0.95	1.13	1.24	1.38	1.42	1.72

Appendix

Table 12: The number of coreference clusters and the average number of mentions (DM: distinct mentions; DNM: distinct named mentions) inside clusters for each size. The DM and DNM values for all clusters are 1.25 and 1.07, respectively.

Mention	A vs. B		A vs. C
Mention	Span	+Type	Span	+Type
All	0.890	0.788	0.867	0.802
Named	0.884	0.825	0.915	0.846
Non-named	0.547	0.547	0.522	0.522

Appendix

Table 13: Inter-annotator agreement (F1 score) for mention annotation. “Span” indicates exact span-only match, and “+Type” indicates exact match for both span and entity type.

Mention	A vs. B		(A or B) vs. LAI
Mention	LEA	CoNLL	LEA	CoNLL
All	0.906	0.939	0.470	0.475
Named	0.909	0.926	0.512	0.496
Non-named	0.603	0.635	0.286	0.395
Mention	A vs. C		(A or C) vs. LAI
Mention	LEA	CoNLL	LEA	CoNLL

Appendix

Table 14: Inter-annotator agreement for coreference annotation.

Type	In/OOKB	A vs. B		A vs. C
Type	In/OOKB	F1	$\kappa$	F1	$\kappa$
Exact	All	0.924	0.919	0.830	0.817
	InKB	0.930	–	0.802	–
	OOKB	0.898	–	0.917	–
Exact	All	0.870	0.866	0.769	0.759
&	InKB	0.881	–	0.767	–

Appendix

Table 15: Inter-annotator agreement for link annotation.

Entity Linking

Entity linking maps mentions of people, places, products, and concepts to the corresponding entries in a knowledge base.

See all 25 papers

LELA: An End-to-end LLM-based Entity Linking Framework with Zero-shot Domain Adaptation
May 26, 2026Samy Haffoudhi, Nikola Dobričić, Fabian Suchanek +1LLM Information Extraction Entity Linking
IP Paris, LTCI, DIG · T´el´ecom Paris, Institut Polytechnique de Paris · IP Paris +1
Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking
Sep 9, 2026Parinthapat Pengpun, Simran Khanuja, Graham NeubigMultimodal Grounding VLM Reasoning
Carnegie Mellon University
J-tau: A Fully Localized Japanese Benchmark for Tool-Agent-User Interaction
Oct 14, 2026Chihiro Yano, Jun Hirako, Hiroaki Yamagiwa +1AI Agent Benchmarks LLM Agent Evaluation
SB Intuitions Corp. · Nagoya University

May 26, 2026J/K move · Enter open · S save

LELA: An End-to-end LLM-based Entity Linking Framework with Zero-shot Domain Adaptation

Samy Haffoudhi, Nikola Dobričić, Fabian Suchanek, Nils Holzenberger

IP Paris, LTCI, DIG · T´el´ecom Paris, Institut Polytechnique de Paris · IP Paris+1

LLM Information Extraction Entity Linking

Open paper AskPDF arXiv

Entity linking is a key component of many downstream NLP systems, yet existing approaches are often tied to the specific target knowledge bases and domains, limiting their real world application. In this paper, we extend LELA, a modular and domain-agnostic LLM-based entity disambiguation method, into a practical Python library that integrates zero-shot Named Entity Recognition (NER) -thereby providing a complete end-toend pipeline for entity-linking in real-world usage. We provide experimental results validating LELA's performance and robustness across diverse entity linking settings. In our demo, users can play with the system on their own input texts.

Abstract

Entity Linking

Explore similar work

LELA: An End-to-end LLM-based Entity Linking Framework with Zero-shot Domain Adaptation

Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking

J-tau: A Fully Localized Japanese Benchmark for Tool-Agent-User Interaction