Controlled Knowledge Engineering for Taxonomy Induction with Tool-Grounded LLM Agents and Runtime-Adaptive Policies
Authors: Jonas Limniatis, Bennet Schaarmann, Georg Reuber, Robert Stahlbock
Organizations: Lufthansa Technik, Hamburg 22335, Germany · Lufthansa Industry Solutions, Norderstedt 22848, Germany · University of Hamburg Business School, Hamburg 20145, Germany · Nordakademie, University of Applied Sciences, Elmshorn 25337, Germany · FOM University of Applied Sciences, Essen 45141, Germany
Taxonomies are structured knowledge representations that support semantic search, recommendation, product catalog organization, and retrieval-augmented generation. Their automated construction from a fixed set of candidate entities requires more than plausible text generation: the resulting hierarchy must be complete, structurally valid, and semantically coherent. Large Language Models (LLMs) are promising for this task, but iterative free-form generation remains vulnerable to omitted or hallucinated entities, invalid hierarchical relations, and inconsistent organizational principles. We introduce TIGA (Taxonomy Induction with Tool-Grounded Agents), a knowledge-engineering architecture that formulates taxonomy induction as an iterative decision process over an explicit state and constrained action space. TIGA combines executable taxonomy-editing tools, an memory that records tool calls, rationales, and outcomes, and a runtime-adaptive organizational policy that the agent revises during inference. An optional global reviewer evaluates the constructed taxonomy against the candidate set and the active policy. We evaluate TIGA on four benchmark taxonomies using two LLM backbones and compare it with single-shot and iterative prompting-based baselines. TIGA achieves the strongest structural-quality results in most evaluated configurations, but does not dominate on every precision measure. The gains are accompanied by substantially higher token consumption, API-call volume, and inference time. Ablation results show consistent benefits from the policy and interaction-memory components, whereas the effect of global review depends on the dataset and the available revision budget. These findings characterize tool grounding and runtime policy adaptation as promising but costly mechanisms for improving the controlled construction of structured knowledge representations.
Figures & tables
Figure 1: Overview of the TIGA inference workflow. Starting from a root concept v0 and a flat candidate set V0 , the agent πθ executes one induction episode over the environment state Si=(Ti,Vi,Pi,Mi) (Section 4.1 ). Each iteration of the agent loop ( 2 – 4 ) assembles the context Ci from a system prompt, the last n messages, and a task message that renders the current state Si ; the LLM selects a single tool action ai∈A (Section 4.2 ), whose successful execution writes back to Ti and Pi . The loop repeats until Vi=∅ (i.e., every candidate entity has been placed), at which the reviewer agent gϕ is being invoked over (Ti,V0,Pi,Mi) (Section 4.3 ). A verdict γi=\textscpass terminates inference and emits the final taxonomy T∗ , whereas γi=\textscrevision prepends the rationale δi to Ci+1 and re-enters the loop for at most k rounds. The reviewer is restricted to aremove and therefore only prunes Ti . Grey monospace labels denote the corresponding middleware hooks of the implementation (Section 5.4 ).
Dataset
#tax
∣V∣
∣E∣
Branch
Max d
WordNet
114
20.5 / 46
19.5 / 45
3.8 / 9.3
3.0 / 3
TExEval-2 †
18
94.2 / 140
94.4 / 140
14.1 / 77.0
3.5 / 5
DBLP †
5
90.8 / 95
89.8 / 94
12.6 / 14.3
2.8 / 3
GProd †
10
90.2 / 127
89.2 / 126
6.8 / 10.7
2.8 / 3
Table 1: Statistics of the evaluation taxonomies. Values are mean / max across the listed number of taxonomies per dataset. Branching denotes children per non-leaf node. Datasets marked † are pruned at depth n from their full versions to fit context-budget constraints.
WordNet
DBLP
TExEval-2
GProd
Model
F1a
F1e
F1a
F1e
F1a
F1e
F1a
F1e
Zero-shot Setting
HF (GPT-5.4-mini)
61.44±1.76
50.32±1.33
13.28±1.07
22.27±2.36
28.07±3.77
28.31±3.15
52.53±5.68
38.31±4.95
HF (Gemma4-31b)
66.83±0.03
55.51±0.11
11.49±0.00
9.45±0.00
24.38±0.35
26.25±0.28
48.01±0.00
42.80±0.00
CoL (GPT-5.4-mini)
73.97±2.23
55.83±1.59
56.08±14.96
39.50±12.96
49.50±5.90
35.68±3.67
52.47±5.75
38.29±4.98
CoL (Gemma4-31b)
80.66±0.97
61.54±0.58
65.21±5.04
40.27±2.62
73.04±2.12
54.86±1.79
81.29±5.33
64.89±4.23
Table 2: Mean F1a and F1e across four benchmarks, both backbones, and both shot settings. Values are mean ±std over 5 random seeds (in percent). Bold indicates the best result per column. Full precision/recall breakdown appears in Table 7 .
Figure 2: Structural quality for F1a and F1e across four backbones, for five TExEval-2 sub-taxonomies ordered by largest deficit TIGA–CoL (5-shot) for Gemma4-31b.
Setting
Pi
Mi
gϕ
F1a
ΔF1a
F1e
ΔF1e
WordNet
– Policy
✗
✓
✓
82.26±0.80
−3.56
62.80±1.00
−4.06
– Memory
✓
✗
✓
82.20±0.50
−3.62
61.30±1.70
−5.56
– Review
✓
✓
✓
86.90±7.60
+1.18
74.30±1.70
+8.04
TExEval-2
– Policy
✗
✓
✓
79.92±2.61
−2.68
60.11±3.32
−4.49
Table 3: Component ablation study of TIGA on WordNet and TExEval-2, using Gemma-4-31b as backbone. Each row either removes or adds one component to method. Values are mean ±std over 5 random seeds; ΔF1a and ΔF1e report the absolute percentage change relative to the baseline.
Figure 3: Distribution of relative update_policy call positions (0 = start, 1 = end of construction), averaged across runs by dataset. Both datasets show a bimodal pattern: an initial spike (policy establishment) and a terminal spike (consolidation), with a lower, approximately flat rate of mid-process correction.
Figure 4: Computational cost–quality trade-off per dataset visualized as a radar plot over four axes: F1e , F1a , total token consumption, and wall-clock inference time in seconds. Larger radial distance indicates better performance on all axes. For tokens and time, values are log-transformed, min–max normalized to [0,1] across all configurations, and inverted ( 1−normalized ) so that cheaper and faster runs map outward. Total tokens include both prompt and completion tokens. Color encodes method (HF, CoL, TIGA); line style encodes backbone model (solid = GPT-5.4-mini, dashed = Gemma4-31b); filled markers denote the 5-shot setting (or TIGA’s default), and hollow markers denote the zero-shot setting.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
#Runs
Mean calls/run
Std
WordNet
597
3.79
1.90
TExEval-2
82
4.54
2.73
Appendix
Table 4: Mean update_policy calls per taxonomy, by dataset
Category
Rule
Hierarchical structure
Taxonomy follows a logical progression from general to specific.
Primary classification
Meat is first divided by processing state (fresh, frozen, tinned, deli) or by animal source (beef, pork).
Processing state vs. animal source
Terms describing meat state (e.g., “frozen meat”) are top-level subcategories of meat .
Terms describing animal source (e.g., “beef,” “pork”) are top-level subcategories of meat .
Sausage classification
Sausage is a distinct category.
Specific sausage types (e.g., “bratwurst,” “salami”) are subcategories of sausage .
Appendix
Table 5: Excerpt of the agent-authored policy Pi for the TExEval-2 meat sub-taxonomy, captured after the first update_policy call. Rules are grouped by organizational category as authored by the agent. This snapshot pairs with Table A.1 , which traces how the policy is revised across the remaining construction process.
#
Update rationale (agent-generated)
1
Initial policy creation to guide the organization of a large batch of meat-related terms, ensuring a logical split between animal types, processing methods, and specific product categories like sausages.
2
The initial batch contains a massive amount of sausages and specific meat cuts. A detailed policy is needed to distinguish between animal-source categories (beef/pork) and product-type categories (sausages), and to handle the specific sub-hierarchy of hot dogs.
3
Formalizing the taxonomy principles based on the extensive organization of the meat, pork, beef, and sausage categories conducted in the previous run.
4
Adding a specific principle to exclude processes from the taxonomy, as “sausage making” was identified as a process and not a product.
Appendix
Table 6: Policy updates obtained by TIGA execution on the TExEval-2 meat sub-taxonomy. Each row corresponds to one update_policy invocation, presented in chronological order.
Dataset
Method
Pa
Ra
F1a
Pe
Re
F1e
WordNet
Zero-shot Setting
HF (GPT-5.4-mini)
79.99±1.08
54.66±1.87
61.44±1.76
60.00±0.95
46.46±1.32
50.32±1.33
HF (Gemma4-31b)
84.01±0.00
60.25±0.06
66.83±0.03
62.29±0.16
50.98±0.09
55.51±0.11
CoL (GPT-5.4-mini)
78.92±2.04
72.16±2.21
73.97±2.23
57.42±1.76
55.28±1.61
55.83±1.59
CoL (Gemma4-31b)
81.91±1.20
81.07±1.04
80.66±0.97
61.85±0.77
61.40±0.58
61.54±0.58
5-shot Setting
Appendix
Table 7: Full precision, recall, and F1 by dataset, method, and shot setting, under both ancestor-level ( Pa , Ra , F1a ) and edge-level ( Pe , Re , F1e ) protocols. Values are mean ±std over 5 random seeds (in percent). Bold indicates the best result per column within each dataset.
Dataset
Method
Backbone
Shot
F1e
F1a
F1en
F1an
Tok. n
Time n
WordNet
HF
GPT-5.4-mini
Zero-shot
50.32
61.44
0.662
0.672
0.573
0.524
HF
GPT-5.4-mini
5-shot
50.63
64.78
0.667
0.717
0.327
0.607
HF
Gemma-4-31b
Zero-shot
55.51
66.83
0.746
0.745
0.313
0.678
HF
Gemma-4-31b
5-shot
55.21
66.55
0.741
0.741
0.685
0.661
CoL
GPT-5.4-mini
Zero-shot
55.83
73.97
0.751
0.841
0.226
0.315
CoL
GPT-5.4-mini
5-shot
54.17
80.66
0.724
0.931
0.210
0.318
Appendix
Table 8: Per-combination F1e and F1a (percent) and the corresponding radar coordinates in [0,1] (superscript n). Further from the center is always better: F1en / F1an are min–max of the raw scores; token and time coordinates are log-transformed, min–max normalized, then reversed ( 1−norm ), so fewer tokens and lower runtime lie outward. Ranges are global across all four datasets. TIGA rows use the baseline configuration (5-shot equivalent).
Table 9: Tool interface specifications exposed to the agent via the LLM tool-calling interface. All tools require a rationale argument, whose value is appended to the memory Mi as part of the tuple (ai,ρi,oi) upon execution. State-mutating tools ( modify ) produce a successor state Si+1=Si on success; the read-only tool ( inspect ) leaves Si unchanged. On failure ( oi=\textscerr ), no structural modification is committed to Ti or Vi ; the error is recorded in Mi+1 .
Taxonomy induction aims to organize concept sets into coherent hierarchical structures. Recent LLM-based methods can induce taxonomies directly from flat term lists, avoiding the need for corpora, but degrade sharply as concept sets scale up. We argue that this degradation stems not only from context length limitations, but also from structural failures in hierarchical reasoning. To address this, we adopt a divide-and-merge paradigm that partitions concepts into smaller subsets, induces local taxonomies, and merges them into a global hierarchy. However, we identify two structural failure modes inherent to this paradigm: Structural Fragmentation, where partitioning weakens local hierarchical signals, and Parent Displacement, where locally plausible relations are misplaced in the global hierarchy. To address both, we propose SPARROW, a scalable taxonomy induction framework that combines structure-preserving spectral partitioning to retain hierarchical connectivity within each block, and constraint-guided incremental fusion that treats block-level relations as structural constraints rather than ground truth for global placement. Experiments on large-scale benchmarks show that SPARROW consistently achieves the strongest global structural quality across backbones. The code is available at https://github.com/rebeccazyr/SPARROW.
Yirui Zhang, Yixuan Tang, Yandong Sun +2
School of Computing, National University of Singapore
Retrieval-Augmented Generation (RAG) helps large language models (LLMs) answer knowledge-intensive and time-sensitive questions by conditioning generation on external evidence. However, most RAG systems still retrieve unstructured chunks and rely on one-shot generation, which often yields redundant context, low information density, and brittle multi-hop reasoning. While structured RAG pipelines can improve grounding, they typically require costly and error-prone graph construction or impose rigid entity-centric structures that do not align with the query's reasoning chain. We propose TaSR-RAG, a taxonomy-guided structured reasoning framework for evidence selection. We represent both queries and documents as relational triples, and constrain entity semantics with a lightweight two-level taxonomy to balance generalization and precision. Given a complex question, TaSR-RAG decomposes it into an ordered sequence of triple sub-queries with explicit latent variables, then performs step-wise evidence selection via hybrid triple matching that combines semantic similarity over raw triples with structural consistency over typed triples. By maintaining an explicit entity binding table across steps, TaSR-RAG resolves intermediate variables and reduces entity conflation without explicit graph construction or exhaustive search. Experiments on multiple multi-hop question answering benchmarks show that TaSR-RAG consistently outperforms strong RAG and structured-RAG baselines by up to 14%, while producing clearer evidence attribution and more faithful reasoning traces.
Recent advances in Large Language Models (LLMs) have demonstrated strong capabilities in generating semantically relevant concepts and relations, making them promising tools for taxonomy enrichment. However, directly relying on LLM-generated expansions often leads to noisy, redundant, or hierarchically inconsistent structures, limiting their reliability for automated taxonomy expansion. In this paper, we present ReLTEx, a framework for reliable LLM-based taxonomy expansion. ReLTEx combines LLM-driven candidate generation with structure-aware validation and recursive expansion control to improve the consistency and quality of generated taxonomies by reducing hallucinations. We evaluate the proposed framework using benchmark taxonomies under a masked taxonomy expansion setting and compare multiple validation strategies. Experimental results, supported by both adapted evaluation metrics and human evaluation, demonstrate that ReLTEx produces more reliable and semantically coherent taxonomy expansions.
Zeinab Ghamlouch, Mehwish Alam
Télécom Paris, Institut Polytechnique de Paris, France