Curriculum-Guided Acquisition of Hierarchical Task Knowledge from LLM-Generated Experience

Yuesheng Xu; Hector Munoz-Avila

cs.AIOct 26, 2026

Curriculum-Guided Acquisition of Hierarchical Task Knowledge from LLM-Generated Experience

Authors: Yuesheng Xu, Hector Munoz-Avila

Organizations: Computer Science & Engineering, Lehigh University; Bethlehem, PA 18015-3084 USA

Abstract

Hierarchical task knowledge enables efficient problem solving by allowing previously acquired skills to be reused across tasks and problem instances. In this paper, we investigate curriculum-guided acquisition of hierarchical task knowledge in an HTN learning framework. We compare three approaches: online learning during problem solving, human-designed curricula, and an automated curriculum generation procedure that derives learning experiences from verified LLM-generated solution traces. The proposed procedure solves small training problems with an LLM, extracts task accomplishment information, and transforms it into ordered sequences of state-task learning problems for HTN method acquisition. We evaluate the three approaches across multiple planning domains. The results demonstrate that automatically generated curricula recover much of the benefit of expert-designed curricula while substantially reducing the token cost of online hierarchical knowledge acquisition.

Figures & tables

Figure 1: Illustration of the automated curriculum generation procedure. (a) Action trace generated by the LLM for a small transportation problem. (b) Corresponding task trace and (task, action-subsequence) mappings indicating the contiguous subsequence of actions over which each task is executed. (c) Hierarchical task structure of the top-level task.

Domain	Curriculum	Small	Medium	Large
Factory	54	442	1042	2082
Mystery	54	406	806	1606
SearchAndRescue	32	308	408	608
Transportation	54	406	806	1606

Table 1: Average number of state axioms per problem, by domain and problem size.

Figure 2: Total token usage across the Factory, Mystery, SearchAndRescue, and Transportation domains. Curriculum-ChatHTN+ incurs an initial overhead during the curriculum learning stage, but this structured approach is essential for acquiring complete domain knowledge. Without it, Vanilla-ChatHTN+ fails on medium and large problems in complex situations. The human-crafted baseline serves as the theoretical upper bound for curriculum learning.

Curriculum Learning

Curriculum learning organizes training examples, tasks, or environments to match how a model learns.

See all 177 papers

Anticipating the Consequences of Curriculum Decisions with Large Language Models
Oct 3, 2026Octavio Pappalardo, Nathan Herr, Tim RocktäschelCurriculum Learning Curriculum RL
University College London
Hierarchical Task Network Planning with LLM-Generated Heuristics
May 8, 2026Felipe Meneguzzi, Alexandre Buchweitz, Augusto B. Corrêa +2Language Model-Guided Search Heuristics Hierarchical Planning
University of Aberdeen, UK · University of Oxford, UK · PUCRS, Brazil +2
Towards LLM-Empowered Knowledge Tracing via LLM-Student Hierarchical Behavior Alignment in Hyperbolic Space
Feb 26, 2026Xingcheng Fu, Shengpeng Wang, Yisen Gao +4Knowledge Tracing Educational Data Mining
Key Lab of Education Blockchain and Intelligent Technology, Guangxi Normal University, Guilin, China · Guangxi Key Lab of Multi-source Information Mining & Security, Guangxi Normal University, Guilin, China · Computer Science and Engineering , The Hong Kong University of Science and Technology, Hong Kong, China +1

Oct 3, 2026J/K move · Enter open · S save

Anticipating the Consequences of Curriculum Decisions with Large Language Models

Octavio Pappalardo, Nathan Herr, Tim Rocktäschel

University College London

Curriculum Learning Curriculum RL LLM-Guided RL

Open paper AskPDF arXiv

Automatic curriculum learning can improve the effectiveness of reinforcement learning by selecting the training experiences presented to the agent over time. Predicting the consequences of such decisions can, however, be difficult. We analyze automatic curriculum learning as a sequential decision-making problem, highlighting a gap between the quantities that determine the value of curriculum decisions and the information captured by local learning signals commonly used to guide them. We then investigate whether Large Language Models (LLMs) can exploit richer information about the learning problem to better anticipate the consequences of curriculum decisions. We introduce a method that combines online learning-progress estimates with LLM-informed estimates of (i) the potential downstream benefits of learning on each task and (ii) whether direct training on a task is currently likely to produce progress. We evaluate the approach on a custom benchmark of 256 textual goals in Craftax under different curriculum objectives. We observe the strongest gains when optimizing for individual target tasks. When optimizing across the full task set, the benefits vary across learners with different mechanisms for cross-task transfer, ranging from modest improvements in learning speed to larger gains that persist through the end of training.

Abstract

Curriculum Learning

Explore similar work

Anticipating the Consequences of Curriculum Decisions with Large Language Models

Hierarchical Task Network Planning with LLM-Generated Heuristics

Towards LLM-Empowered Knowledge Tracing via LLM-Student Hierarchical Behavior Alignment in Hyperbolic Space