Controlled Knowledge Engineering for Taxonomy Induction with Tool-Grounded LLM Agents and Runtime-Adaptive Policies
Authors: Jonas Limniatis, Bennet Schaarmann, Georg Reuber, Robert Stahlbock
Organizations: Lufthansa Technik, Hamburg 22335, Germany · Lufthansa Industry Solutions, Norderstedt 22848, Germany · University of Hamburg Business School, Hamburg 20145, Germany · Nordakademie, University of Applied Sciences, Elmshorn 25337, Germany · FOM University of Applied Sciences, Essen 45141, Germany
Taxonomies are structured knowledge representations that support semantic search, recommendation, product catalog organization, and retrieval-augmented generation. Their automated construction from a fixed set of candidate entities requires more than plausible text generation: the resulting hierarchy must be complete, structurally valid, and semantically coherent. Large Language Models (LLMs) are promising for this task, but iterative free-form generation remains vulnerable to omitted or hallucinated entities, invalid hierarchical relations, and inconsistent organizational principles. We introduce TIGA (Taxonomy Induction with Tool-Grounded Agents), a knowledge-engineering architecture that formulates taxonomy induction as an iterative decision process over an explicit state and constrained action space. TIGA combines executable taxonomy-editing tools, an memory that records tool calls, rationales, and outcomes, and a runtime-adaptive organizational policy that the agent revises during inference. An optional global reviewer evaluates the constructed taxonomy against the candidate set and the active policy. We evaluate TIGA on four benchmark taxonomies using two LLM backbones and compare it with single-shot and iterative prompting-based baselines. TIGA achieves the strongest structural-quality results in most evaluated configurations, but does not dominate on every precision measure. The gains are accompanied by substantially higher token consumption, API-call volume, and inference time. Ablation results show consistent benefits from the policy and interaction-memory components, whereas the effect of global review depends on the dataset and the available revision budget. These findings characterize tool grounding and runtime policy adaptation as promising but costly mechanisms for improving the controlled construction of structured knowledge representations.
Figures & tables
Figure 1: Overview of the TIGA inference workflow. Starting from a root concept v0 and a flat candidate set V0 , the agent πθ executes one induction episode over the environment state Si=(Ti,Vi,Pi,Mi) (Section 4.1 ). Each iteration of the agent loop ( 2 – 4 ) assembles the context Ci from a system prompt, the last n messages, and a task message that renders the current state Si ; the LLM selects a single tool action ai∈A (Section 4.2 ), whose successful execution writes back to Ti and Pi . The loop repeats until Vi=∅ (i.e., every candidate entity has been placed), at which the reviewer agent gϕ is being invoked over (Ti,V0,Pi,Mi) (Section 4.3 ). A verdict γi=\textscpass terminates inference and emits the final taxonomy T∗ , whereas γi=\textscrevision prepends the rationale δi to Ci+1 and re-enters the loop for at most k rounds. The reviewer is restricted to aremove and therefore only prunes Ti . Grey monospace labels denote the corresponding middleware hooks of the implementation (Section 5.4 ).
Dataset
#tax
∣V∣
∣E∣
Branch
Max d
WordNet
114
20.5 / 46
19.5 / 45
3.8 / 9.3
3.0 / 3
TExEval-2 †
18
94.2 / 140
94.4 / 140
14.1 / 77.0
3.5 / 5
DBLP †
5
90.8 / 95
89.8 / 94
12.6 / 14.3
2.8 / 3
GProd †
10
90.2 / 127
89.2 / 126
6.8 / 10.7
2.8 / 3
Table 1: Statistics of the evaluation taxonomies. Values are mean / max across the listed number of taxonomies per dataset. Branching denotes children per non-leaf node. Datasets marked † are pruned at depth n from their full versions to fit context-budget constraints.
WordNet
DBLP
TExEval-2
GProd
Model
F1a
F1e
F1a
F1e
F1a
F1e
F1a
F1e
Zero-shot Setting
HF (GPT-5.4-mini)
61.44±1.76
50.32±1.33
13.28±1.07
22.27±2.36
28.07±3.77
28.31±3.15
52.53±5.68
38.31±4.95
HF (Gemma4-31b)
66.83±0.03
55.51±0.11
11.49±0.00
9.45±0.00
24.38±0.35
26.25±0.28
48.01±0.00
42.80±0.00
CoL (GPT-5.4-mini)
73.97±2.23
55.83±1.59
56.08±14.96
39.50±12.96
49.50±5.90
35.68±3.67
52.47±5.75
38.29±4.98
CoL (Gemma4-31b)
80.66±0.97
61.54±0.58
65.21±5.04
40.27±2.62
73.04±2.12
54.86±1.79
81.29±5.33
64.89±4.23
Table 2: Mean F1a and F1e across four benchmarks, both backbones, and both shot settings. Values are mean ±std over 5 random seeds (in percent). Bold indicates the best result per column. Full precision/recall breakdown appears in Table 7 .
Figure 2: Structural quality for F1a and F1e across four backbones, for five TExEval-2 sub-taxonomies ordered by largest deficit TIGA–CoL (5-shot) for Gemma4-31b.
Setting
Pi
Mi
gϕ
F1a
ΔF1a
F1e
ΔF1e
WordNet
– Policy
✗
✓
✓
82.26±0.80
−3.56
62.80±1.00
−4.06
– Memory
✓
✗
✓
82.20±0.50
−3.62
61.30±1.70
−5.56
– Review
✓
✓
✓
86.90±7.60
+1.18
74.30±1.70
+8.04
TExEval-2
– Policy
✗
✓
✓
79.92±2.61
−2.68
60.11±3.32
−4.49
Table 3: Component ablation study of TIGA on WordNet and TExEval-2, using Gemma-4-31b as backbone. Each row either removes or adds one component to method. Values are mean ±std over 5 random seeds; ΔF1a and ΔF1e report the absolute percentage change relative to the baseline.
Figure 3: Distribution of relative update_policy call positions (0 = start, 1 = end of construction), averaged across runs by dataset. Both datasets show a bimodal pattern: an initial spike (policy establishment) and a terminal spike (consolidation), with a lower, approximately flat rate of mid-process correction.
Figure 4: Computational cost–quality trade-off per dataset visualized as a radar plot over four axes: F1e , F1a , total token consumption, and wall-clock inference time in seconds. Larger radial distance indicates better performance on all axes. For tokens and time, values are log-transformed, min–max normalized to [0,1] across all configurations, and inverted ( 1−normalized ) so that cheaper and faster runs map outward. Total tokens include both prompt and completion tokens. Color encodes method (HF, CoL, TIGA); line style encodes backbone model (solid = GPT-5.4-mini, dashed = Gemma4-31b); filled markers denote the 5-shot setting (or TIGA’s default), and hollow markers denote the zero-shot setting.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
#Runs
Mean calls/run
Std
WordNet
597
3.79
1.90
TExEval-2
82
4.54
2.73
Appendix
Table 4: Mean update_policy calls per taxonomy, by dataset
Category
Rule
Hierarchical structure
Taxonomy follows a logical progression from general to specific.
Primary classification
Meat is first divided by processing state (fresh, frozen, tinned, deli) or by animal source (beef, pork).
Processing state vs. animal source
Terms describing meat state (e.g., “frozen meat”) are top-level subcategories of meat .
Terms describing animal source (e.g., “beef,” “pork”) are top-level subcategories of meat .
Sausage classification
Sausage is a distinct category.
Specific sausage types (e.g., “bratwurst,” “salami”) are subcategories of sausage .
Appendix
Table 5: Excerpt of the agent-authored policy Pi for the TExEval-2 meat sub-taxonomy, captured after the first update_policy call. Rules are grouped by organizational category as authored by the agent. This snapshot pairs with Table A.1 , which traces how the policy is revised across the remaining construction process.
#
Update rationale (agent-generated)
1
Initial policy creation to guide the organization of a large batch of meat-related terms, ensuring a logical split between animal types, processing methods, and specific product categories like sausages.
2
The initial batch contains a massive amount of sausages and specific meat cuts. A detailed policy is needed to distinguish between animal-source categories (beef/pork) and product-type categories (sausages), and to handle the specific sub-hierarchy of hot dogs.
3
Formalizing the taxonomy principles based on the extensive organization of the meat, pork, beef, and sausage categories conducted in the previous run.
4
Adding a specific principle to exclude processes from the taxonomy, as “sausage making” was identified as a process and not a product.
Appendix
Table 6: Policy updates obtained by TIGA execution on the TExEval-2 meat sub-taxonomy. Each row corresponds to one update_policy invocation, presented in chronological order.
Dataset
Method
Pa
Ra
F1a
Pe
Re
F1e
WordNet
Zero-shot Setting
HF (GPT-5.4-mini)
79.99±1.08
54.66±1.87
61.44±1.76
60.00±0.95
46.46±1.32
50.32±1.33
HF (Gemma4-31b)
84.01±0.00
60.25±0.06
66.83±0.03
62.29±0.16
50.98±0.09
55.51±0.11
CoL (GPT-5.4-mini)
78.92±2.04
72.16±2.21
73.97±2.23
57.42±1.76
55.28±1.61
55.83±1.59
CoL (Gemma4-31b)
81.91±1.20
81.07±1.04
80.66±0.97
61.85±0.77
61.40±0.58
61.54±0.58
5-shot Setting
Appendix
Table 7: Full precision, recall, and F1 by dataset, method, and shot setting, under both ancestor-level ( Pa , Ra , F1a ) and edge-level ( Pe , Re , F1e ) protocols. Values are mean ±std over 5 random seeds (in percent). Bold indicates the best result per column within each dataset.
Dataset
Method
Backbone
Shot
F1e
F1a
F1en
F1an
Tok. n
Time n
WordNet
HF
GPT-5.4-mini
Zero-shot
50.32
61.44
0.662
0.672
0.573
0.524
HF
GPT-5.4-mini
5-shot
50.63
64.78
0.667
0.717
0.327
0.607
HF
Gemma-4-31b
Zero-shot
55.51
66.83
0.746
0.745
0.313
0.678
HF
Gemma-4-31b
5-shot
55.21
66.55
0.741
0.741
0.685
0.661
CoL
GPT-5.4-mini
Zero-shot
55.83
73.97
0.751
0.841
0.226
0.315
CoL
GPT-5.4-mini
5-shot
54.17
80.66
0.724
0.931
0.210
0.318
Appendix
Table 8: Per-combination F1e and F1a (percent) and the corresponding radar coordinates in [0,1] (superscript n). Further from the center is always better: F1en / F1an are min–max of the raw scores; token and time coordinates are log-transformed, min–max normalized, then reversed ( 1−norm ), so fewer tokens and lower runtime lie outward. Ranges are global across all four datasets. TIGA rows use the baseline configuration (5-shot equivalent).
Table 9: Tool interface specifications exposed to the agent via the LLM tool-calling interface. All tools require a rationale argument, whose value is appended to the memory Mi as part of the tuple (ai,ρi,oi) upon execution. State-mutating tools ( modify ) produce a successor state Si+1=Si on success; the read-only tool ( inspect ) leaves Si unchanged. On failure ( oi=\textscerr ), no structural modification is committed to Ti or Vi ; the error is recorded in Mi+1 .