Skip to main content
Back to timeline
arXivSource publication:

Taxonomy building stops being free-form text: TIGA turns each edit into an executable tool call, leading recall and F1 in most settings

Synopsis

TIGA recasts building a taxonomy from a fixed candidate entity set as an iterative decision process over an explicit state: the agent may only invoke executable editing tools, and it maintains a memory of actions, rationales and outcomes plus an organizational policy it revises during inference. Across four benchmark taxonomies and two model backbones it achieves the strongest structural quality in most configurations, with the clearest advantages in recall and F1, but it does not lead on every precision measure and consumes substantially more tokens, API calls and inference time.

Source-provided article image: Controlled Knowledge Engineering for Taxonomy Induction with Tool-Grounded LLM Agents and Runtime-Adaptive Policies
arXiv

Interpretation

Taxonomy construction moves from generating a whole hierarchy as text in one pass to executing constrained editing operations step by step over an explicit state, so each change is a verifiable tool call and intermediate states are inspectable. Earlier prompting-based methods kept reasoning, decisions and structural modification inside generated text that was later parsed or filtered, so intermediate decisions were not retained as verifiable state. On the four benchmarks WordNet, DBLP, TExEval-2 and Google Product Taxonomy, TIGA achieves the highest recall or F1 in 7 of 8 reported settings; relative to the strongest baseline in each setting, it improves by 8.1 percentage points on Google Product Taxonomy and by 7.9 and 7.4 percentage points on DBLP.

The agent proposes one operation, and a tool layer executes it against the current taxonomy and returns explicit feedback, separating semantic judgment from structural modification. The tool set is deliberately kept small, split into state-mutating entity addition, entity removal and policy update plus a read-only subtree view, and every call must carry a rationale. On TExEval-2 with GPT-5.4-mini, constructing 18 sub-taxonomies issues 1008.2 API calls, versus 160.8 for CoL in the 5-shot setting, about 6.3 times as many.

Ablations show the dynamic policy and interaction memory contribute consistently to structural quality on the evaluated benchmarks, while the effect of global review depends on the dataset and the revision budget. Removing the policy lowers the metric by 3.6% on WordNet and 2.7% on TExEval-2; removing memory lowers edge-level F1 by 5.6% and 5.8% respectively. Review raises edge-level F1 by 8.0% on WordNet but lowers both metrics on TExEval-2 by 1.4% and 0.8%, with revision capped at three rounds.

Perspective

Evaluation covers the four benchmarks WordNet, DBLP, TExEval-2 and Google Product Taxonomy with the GPT-5.4-mini and Gemma4-31b backbones, each configuration run five independent times; Google Product Taxonomy is depth-truncated and sampled into 10 sub-taxonomies, and TExEval-2 provides no benchmark-specific few-shot examples.

TIGA does not dominate every metric: it trails baselines on some precision measures for WordNet, DBLP and Google Product Taxonomy, with the largest gap 13.5 percentage points on DBLP; run-to-run variability is larger on Google Product Taxonomy, where under GPT-5.4-mini the standard deviations for F1 and recall are 8.5% and 7.9% versus 3.5% and 2.8% for CoL. The authors attribute this variability to early errors propagating over a longer interaction sequence in a larger candidate space, but state that the experiments do not directly isolate its causal source; under the three-round cap some taxonomies still fail the reviewer verdict and are returned unmodified, and a reviewer verdict is not a guarantee of correctness.