Anticipate, Adapt, Act: A Hybrid Framework for Task Planning

Nabanita Dash; Ayush Kaura; Shivam Singh; Ramandeep Singh; Snehasis Banerjee; Mohan Sridharan; K. Madhava Krishna

cs.ROFeb 23, 2026

Anticipate, Adapt, Act: A Hybrid Framework for Task Planning

Authors: Nabanita Dash, Ayush Kaura, Shivam Singh, Ramandeep Singh, Snehasis Banerjee, Mohan Sridharan, K. Madhava Krishna

Organizations: Robotics Research Center, IIIT Hyderabad, India · TCS Research, Tata Consultancy Services, India · School of Informatics, University of Edinburgh, UK

Abstract

Anticipating and adapting to failures is a key capability robots need to collaborate effectively with humans in complex domains. This continues to be a challenge despite the impressive performance of state of the art AI planning systems and Large Language Models (LLMs) because of the uncertainty associated with the tasks and their outcomes. Toward addressing this challenge, we present a hybrid framework that integrates the generic prediction capabilities of an LLM with the probabilistic sequential decision-making capability of Relational Dynamic Influence Diagram Language. For any given task, the robot reasons about the task and the capabilities of the human attempting to complete it; predicts potential failures due to lack of ability (in the human) or lack of relevant domain objects; and executes actions to prevent such failures or recover from them. Experimental evaluation in the VirtualHome 3D simulation environment demonstrates substantial improvement in performance compared with state of the art baselines.

Figures & tables

Fig. 1 : Illustrative task of fetching a glass of water from the sink to the kitchen counter. In the baseline scenario, the human may end up dropping the water glass due to stability issues. Our framework enables the robot to anticipate such failures; it either prevents the failure by completing the task, or prepares to recover from the failure by fetching a mop that can be used to clean the potential water spill.

Fig. 2 : Our framework’s pipeline: (a) LLM takes a prompt of task lists, user preferences, scene description, and current command, to predict upcoming tasks; (b) RDDL description of domain knowledge and joint goals comprising current and predicted tasks fed to PROST planner; and (c) Plan of actions to be executed by robot and human to achieve the goal, including robot’s actions to prevent or recover from potential failures due to the human’s actions.

Fig. 3 : Partial description of reward specification for the

PrepareBreakfast(Toast)

task.

Fig. 4 : Example probabilities of state transition probabilities in human behavior model. The probability of the human: walking to the kitchen from a random location is 0.8; grabbing a bread slice while in the kitchen is 0.625; placing the bread slice in the toaster after grabbing it is 0.6; switching on the toaster while in the kitchen is 0.625.

Task	Subgoals completion %			Task completion %
	(LLM)	(RDDL)	(Ours)	(LLM)	(RDDL)	(Ours)
Prepare and Serve Salmon + Water	46.67%	63.3%	85%	30%	55%	80%
Prepare and Serve Coffee + Wash Dish	47.5%	75%	86.7%	20%	55%	87.5%
Prepare and Serve Cereal + Coffee	52.5%	70%	88.3%	25%	60%	85%
Prepare and Serve Toast + Coffee	42.5%	68.3%	83.3%	15%	58.3%	84%
Prepare and Serve Pizza + Wash Dish	35.56%	66.7%	84.2%	10%	57%	83.4%

TABLE I : Task and subgoal completion performance of our framework is substantially better than that of the two baselines, LLM-only and RDDL/PROST without anticipatory rewards, over selected composite tasks.

Task	Failures			Prevention			Recovery			Avg. Actions
	L	R	O	L	R	O	L	R	O	L	R	O
Prepare and Serve Salmon + Water	18/30	17/30	14/30	12/30	13/30	16/30	6/18	0	11/14	–	–	38
Prepare and Serve Coffee + Wash Dish	9/30	11/30	8/30	21/30	19/30	22/30	0	2/11	6/8	24	–	26
Prepare and Serve Cereal + Coffee	25/30	13/30	10/30	5/30	17/30	20/30	9/25	0	8/10	–	–	42
Prepare and Serve Toast + Coffee	27/30	9/30	12/30	3/30	21/30	18/30	6/27	0	9/12	–	–	48
Prepare and Serve Pizza + Wash Dish	18/30	11/30	9/30	12/30	19/30	21/30	3/18	0	7/9	–	–	30

TABLE II : Failure prevention and recovery statistics for selected composite tasks. Prevention indicates proactive avoidance of likely human failures, while Recovery refers to corrective action after failure has occurred. In most scenarios, the LLM-only baseline ( L ) or RDDL/PROST baseline ( R ) failed to complete the composite tasks, with Avg. Actions and Time Taken reported as ‘–’. Our framework ( O ) consistently completes tasks through anticipation and recovery.

Fig. 5 : Task failures as a function of the reward-based multiplier that determines extent to which failure prevention and recovery is prioritized over task completion. With an increase in the multiplier, failures reduce up to a point before increasing again. A trade off between failure prevention and task completion leads to good performance.

Explore similar work

HEART: Coordination of Heterogeneous Expert Agents for Physically Grounded Robotic Task Planning
Jun 24, 2026Junho Lee, Seabin Lee, Wonjong Lee +3Large Language Model-Based Robot Planning Robot Task Planning
Dept. of Electronic Engineering, Sogang University, Seoul, Korea
On-Demand Human Assistance for Task Continuation under Physical Action Failures in LLM-based Planning
Mar 30, 2026Shoichi Hasegawa, Akira Taniguchi, Lotfi El Hafi +2Large Language Model-Based Robot Planning Robot Failure Recovery
Shoichi Hasegawa and Akira Taniguchi are with Ritsumeikan University; 2-150 Iwakura, Ibaraki, Osaka 567-8570, Japan. · Akira Taniguchi is with Kyushu Institute of Technology; 680-4 Kawadu, Iizuka, Fukuoka 820-0067, Japan. · Lotfi El Hafi is with Coarobo GK; 2-2-2 Hikaridai, Seika, Soura, Kyoto 619-0237, Japan. +4
Agentic Harnesses: LLM-Driven Verification Layers for Robot Autonomy
Aug 10, 2026Rohan Bhagra, Mahantesh Halapannavar, Uddhav BhattaraiLarge Language Model-Based Robot Planning Adversarial Robustness
Carnegie Mellon University, Department of Information Systems

Jun 24, 2026J/K move · Enter open · S save

HEART: Coordination of Heterogeneous Expert Agents for Physically Grounded Robotic Task Planning

Junho Lee, Seabin Lee, Wonjong Lee, Nayoung Kim+2

Dept. of Electronic Engineering, Sogang University, Seoul, Korea

Large Language Model-Based Robot Planning Robot Task Planning Multi-Agent Planning Multi-Agent LLMs LLM Agent Planning

Open paper AskPDF arXiv

Large Language Models (LLMs) can reason over complex instructions but often fail to satisfy the physical and spatial constraints required for robotic task planning. Recent LLM-based planners directly translate text into action sequences, yet they lack structured reasoning about feasibility, reachability, and logical order, resulting in invalid or incomplete plans. We present a heterogeneous multi-LLM framework that decomposes instructions into atomic reasoning tasks and allocates them to role-specialized expert agents under a token budget for real-world computational and communicational constraints. By combining role-oriented reasoning from heterogeneous agents followed by constraint-driven plan synthesis, HEART validates capability, reachability, and constraint conditions before planning and helps produce physically executable plans while maintaining efficiency. Experiments across different household benchmarks show that HEART consistently improves plan success compared to single-LLM and rule-based planners, demonstrating that heterogeneous LLM collaboration enables robust and scalable robotic task planning under resource constraints.

Abstract

Explore similar work

HEART: Coordination of Heterogeneous Expert Agents for Physically Grounded Robotic Task Planning

On-Demand Human Assistance for Task Continuation under Physical Action Failures in LLM-based Planning

Agentic Harnesses: LLM-Driven Verification Layers for Robot Autonomy